code-server/patches/telemetry.diff

164 lines
7.0 KiB
Diff
Raw Normal View History

Add support for telemetry endpoint
To test:
chore: upgrade Code to 1.74.1 (#5909) * chore: upgrade Code to 1.74.1 * chore: remove require in integration.diff I don't know what the impact of this is but in https://github.com/microsoft/vscode/commit/192c67db71e8c261f26e2f34c86a4791ae428b2f they removed the usage of `require` in `server.main.ts`. More details in PR: https://github.com/microsoft/vscode/pull/165831 * chore: update marketplace.diff * chore: update sha hash in webview.diff * chore: update disable-builtin-ext-update.diff If my logic is right, then this patch is now simplified thanks to this: https://github.com/microsoft/vscode/blob/1.74.1/src/vs/workbench/contrib/extensions/browser/extensionsWorkbenchService.ts#L1238 * chore: refresh proxy-uri patch * chore: refresh local-storage.diff * chore: refresh sourcemaps.diff * chore: refresh disable-downloads.diff * chore: refresh display-language.diff * chore: refresh getting-started.diff * docs: update testing notes for cli-window-open * docs: update telemetry testing instructions * fix: add GITHUB_TOKEN to build code-server job Downloading @vscode/ripgrep is failing only in CI so adding this environment variable to see if it increases the rate limit. Ref: https://github.com/microsoft/vscode-ripgrep#github-api-limit-note * refactor: use own cache key build code-server job * temp: disable vscode test * refactor: delete wrapper test * Revert "refactor: delete wrapper test" This reverts commit 3999279b73c3519c7dbb03dfc7076bf26f717e13. * refactor: move vscode tests to e2e (#5911) * wip: migrate vscode tests to e2e * feat: add codeWorkspace to global setup * refactor: only use dir in spawn when we should * wip: migrate more tests * refactor: move all vscode tests to e2e * refactor(ci): move unit to own job * fixup: add codecov to unit test step * Update test/e2e/models/CodeServer.ts * Update test/e2e/models/CodeServer.ts * docs: add note about intercept requests * refactor: rm unused clean() calls * refactor: delete duplicate test * refactor: update 'should not redirect' test * refactor: rm unused imports * refactor: rm unnecessary navigate call in test * fixup: formatting * wip: update test * refactor: modify assertion for proxy * fixup: use REVERSE_PROXY_BASE_PATH * refactor: add helper fn getMaybeProxiedPathname * fixup: formatting * fixup: rm unused import * chore: increase playwright timeout * Revert "chore: increase playwright timeout" This reverts commit a059129252216c5f5cba83e9bca3d90cf658b7be. * chore: rm timeout
2022-12-23 01:25:28 +08:00
1. Create a mock API using [RequestBin](https://requestbin.io/) or [Beeceptor](https://beeceptor.com/)
2. Run code-server with `CS_TELEMETRY_URL` set:
i.e. `CS_TELEMETRY_URL="https://requestbin.io/1ebub9z1" ./code-server-<version>-macos-amd64/bin/code-server`
chore: upgrade Code to 1.73.0 (#5751) * chore: upgrade Code to 1.73.0 This upgrades Code to 1.73.0 via the tag. * chore: refresh integration patch * chore: clean up base-path patch Only change here was they moved lib/vscode/src/vs/platform/extensionResourceLoader/common/extensionResourceLoader.ts so I had to update it. Code still looks the same though. * chore: refresh proposed-api patch * chore: update marketplace patch Simlar to a previous patch, the location of lib/vscode/src/vs/platform/extensionResourceLoader/common/extensionResourceLoader.ts changed so I had to update this patch. No changes to code itself. * chore: update hash in webview patch I believe there was only one to update but I may have missed one. * chore: refresh disable-builtin-ext-update.diff * chore: refresh update-check quilt couldn't apply it so I had to add one change in manually to lib/vscode/src/vs/server/node/serverEnvironmentService.ts * chore: refresh logout patch * chore: refresh proxy-uri patch * chore: refresh local-storage patch * chore: refresh sourcemaps patch * chore: refresh disable-downloads patch * chore: refresh telemetry patch * refactor: re-apply display-language patch This kinda got removed but I added it back in. * refactor: drop exec-argv patch This was accepted upstream! :tada * chore: refresh getting-started patch * fixup: add missing slash in marketplace * fixup: update notes proposed-api patch * fixup: support this.args.log as string Seems like upstream now uses a string[] for this. For now, support string. See https://github.com/microsoft/vscode/commit/2b50ab06b1636a38f6bec3dfb2c8f471374a2cba * Revert "fixup: support this.args.log as string" This reverts commit 78c02a1f137655e27f3137e1d07a274e482baf6b. * fixup!: add log to toCodeArgs This was changed upstream from `string` to `string[]` so now we convert to an array in `toCodeArgs`. See https://github.com/coder/code-server/pull/5751/commits/78c02a1f137655e27f3137e1d07a274e482baf6b * fixup: update telemetry description
2022-11-10 06:10:03 +08:00
NOTE: it has to be a production build.
3. Load code-server in browser an do things (i.e. open a file)
4. Refresh RequestBin and you should see logs
Index: code-server/lib/vscode/src/vs/server/node/serverServices.ts
===================================================================
--- code-server.orig/lib/vscode/src/vs/server/node/serverServices.ts
+++ code-server/lib/vscode/src/vs/server/node/serverServices.ts
@@ -65,6 +65,7 @@ import { IExtensionsScannerService } fro
chore: update Code to 1.68 (#5263) * chore: update Code to 1.67 Was able to remove our changes to common/webview.ts since they are upstream now. Other than that no serious changes, just context diffs. * chore: update Code to 1.68 - Upstream moved the web socket endpoint so change the Express route from / to *. That will let web sockets work at any endpoint. - Everything in the workbench config is basically the same but de-indented (upstream extracted it into a separate object which resulted in a de-indent), the ordering is slightly different, and instead of vscodeBase we now need vscodeBase + this._staticRoute since everything is served from a sub-path now. - Move manifest link back to the root since that is where we host our manifest. - Change RemoteAuthoritiesImpl to use the same path building method as in other places (+ instead of using URI.parse/join). - Use existing host/port in RemoteAuthoritiesImpl and BrowserSocketFactory instead of patching them to use window.location (these are set from window.location to begin with so it should be the same result but with less patching). - Since BrowserSocketFactory includes a sub-path now (endpoints were changed upstream to serve from /quality/commit instead of from the root) the patch there has changed to prepend the base to that path (instead of using the base directly). - The workbench HTML now natively supports a base URL in the form of WORKBENCH_WEB_BASE_URL so no need for VS_BASE patches there anymore. - Upstream added type="image/x-icon" so I did as well. - Move the language patch to the end of the series so it is easier to eventually remove. - Remove the existing NLS config in favor of one that supports extensions. - Upstream deleted webview main.js and inlined it into the HTML so move that code (the parent origin check) into both those HTML files (index.html and index-no-csp.html). - The remaining diff is from changes to the surrounding context or a line was changed slightly by upstream (for example renamed files or new arguments like to the remote authority resolver). * fix: modify product.json before building Code injects this into the client during the build process so it needs to be updated before we build. * fix: update inline script nonces * Update HTML base path test * fix: missing commit Code overrides it with nothing. The date is also already injected. * fix: web extensions breaking when the commit changes By just using the marketplace directly instead of going through the backend. I am not sure what the point is when searching extensions already goes directly to the marketplace anyway. But also remove the prefix that breaks this as well because otherwise existing installations will break.
2022-06-22 05:51:46 +08:00
import { ExtensionsScannerService } from 'vs/server/node/extensionsScannerService';
import { IExtensionsProfileScannerService } from 'vs/platform/extensionManagement/common/extensionsProfileScannerService';
import { IUserDataProfilesService } from 'vs/platform/userDataProfile/common/userDataProfile';
+import { TelemetryClient } from 'vs/server/node/telemetryClient';
chore: update Code to 1.68 (#5263) * chore: update Code to 1.67 Was able to remove our changes to common/webview.ts since they are upstream now. Other than that no serious changes, just context diffs. * chore: update Code to 1.68 - Upstream moved the web socket endpoint so change the Express route from / to *. That will let web sockets work at any endpoint. - Everything in the workbench config is basically the same but de-indented (upstream extracted it into a separate object which resulted in a de-indent), the ordering is slightly different, and instead of vscodeBase we now need vscodeBase + this._staticRoute since everything is served from a sub-path now. - Move manifest link back to the root since that is where we host our manifest. - Change RemoteAuthoritiesImpl to use the same path building method as in other places (+ instead of using URI.parse/join). - Use existing host/port in RemoteAuthoritiesImpl and BrowserSocketFactory instead of patching them to use window.location (these are set from window.location to begin with so it should be the same result but with less patching). - Since BrowserSocketFactory includes a sub-path now (endpoints were changed upstream to serve from /quality/commit instead of from the root) the patch there has changed to prepend the base to that path (instead of using the base directly). - The workbench HTML now natively supports a base URL in the form of WORKBENCH_WEB_BASE_URL so no need for VS_BASE patches there anymore. - Upstream added type="image/x-icon" so I did as well. - Move the language patch to the end of the series so it is easier to eventually remove. - Remove the existing NLS config in favor of one that supports extensions. - Upstream deleted webview main.js and inlined it into the HTML so move that code (the parent origin check) into both those HTML files (index.html and index-no-csp.html). - The remaining diff is from changes to the surrounding context or a line was changed slightly by upstream (for example renamed files or new arguments like to the remote authority resolver). * fix: modify product.json before building Code injects this into the client during the build process so it needs to be updated before we build. * fix: update inline script nonces * Update HTML base path test * fix: missing commit Code overrides it with nothing. The date is also already injected. * fix: web extensions breaking when the commit changes By just using the marketplace directly instead of going through the backend. I am not sure what the point is when searching extensions already goes directly to the marketplace anyway. But also remove the prefix that breaks this as well because otherwise existing installations will break.
2022-06-22 05:51:46 +08:00
import { NullPolicyService } from 'vs/platform/policy/common/policy';
import { OneDataSystemAppender } from 'vs/platform/telemetry/node/1dsAppender';
chore: upgrade Code to 1.73.0 (#5751) * chore: upgrade Code to 1.73.0 This upgrades Code to 1.73.0 via the tag. * chore: refresh integration patch * chore: clean up base-path patch Only change here was they moved lib/vscode/src/vs/platform/extensionResourceLoader/common/extensionResourceLoader.ts so I had to update it. Code still looks the same though. * chore: refresh proposed-api patch * chore: update marketplace patch Simlar to a previous patch, the location of lib/vscode/src/vs/platform/extensionResourceLoader/common/extensionResourceLoader.ts changed so I had to update this patch. No changes to code itself. * chore: update hash in webview patch I believe there was only one to update but I may have missed one. * chore: refresh disable-builtin-ext-update.diff * chore: refresh update-check quilt couldn't apply it so I had to add one change in manually to lib/vscode/src/vs/server/node/serverEnvironmentService.ts * chore: refresh logout patch * chore: refresh proxy-uri patch * chore: refresh local-storage patch * chore: refresh sourcemaps patch * chore: refresh disable-downloads patch * chore: refresh telemetry patch * refactor: re-apply display-language patch This kinda got removed but I added it back in. * refactor: drop exec-argv patch This was accepted upstream! :tada * chore: refresh getting-started patch * fixup: add missing slash in marketplace * fixup: update notes proposed-api patch * fixup: support this.args.log as string Seems like upstream now uses a string[] for this. For now, support string. See https://github.com/microsoft/vscode/commit/2b50ab06b1636a38f6bec3dfb2c8f471374a2cba * Revert "fixup: support this.args.log as string" This reverts commit 78c02a1f137655e27f3137e1d07a274e482baf6b. * fixup!: add log to toCodeArgs This was changed upstream from `string` to `string[]` so now we convert to an array in `toCodeArgs`. See https://github.com/coder/code-server/pull/5751/commits/78c02a1f137655e27f3137e1d07a274e482baf6b * fixup: update telemetry description
2022-11-10 06:10:03 +08:00
import { LoggerService } from 'vs/platform/log/node/loggerService';
@@ -147,11 +148,23 @@ export async function setupServerService
2024-05-07 07:53:53 +08:00
const requestService = new RequestService(configurationService, environmentService, logService, loggerService);
services.set(IRequestService, requestService);
+ let isContainer = undefined;
+ try {
+ await Promises.stat('/run/.containerenv');
+ isContainer = true;
+ } catch (error) { /* Does not exist, probably. */ }
+ if (!isContainer) {
+ try {
+ const content = await Promises.readFile('/proc/self/cgroup', 'utf8')
+ isContainer = content.includes('docker');
+ } catch (error) { /* Permission denied, probably. */ }
+ }
+
let oneDsAppender: ITelemetryAppender = NullAppender;
const isInternal = isInternalTelemetry(productService, configurationService);
if (supportsTelemetry(productService, environmentService)) {
- if (!isLoggingOnly(productService, environmentService) && productService.aiConfig?.ariaKey) {
2024-05-07 11:27:34 +08:00
- oneDsAppender = new OneDataSystemAppender(requestService, isInternal, eventPrefix, null, productService.aiConfig.ariaKey);
+ if (!isLoggingOnly(productService, environmentService) && productService.telemetryEndpoint) {
2024-05-08 03:33:03 +08:00
+ oneDsAppender = new OneDataSystemAppender(requestService, isInternal, eventPrefix, null, () => new TelemetryClient(productService.telemetryEndpoint!, machineId, isContainer));
disposables.add(toDisposable(() => oneDsAppender?.flush())); // Ensure the AI appender is disposed so that it flushes remaining data
}
2024-05-07 11:27:34 +08:00
Index: code-server/lib/vscode/src/vs/server/node/telemetryClient.ts
===================================================================
--- /dev/null
+++ code-server/lib/vscode/src/vs/server/node/telemetryClient.ts
@@ -0,0 +1,71 @@
+import { AppInsightsCore, IExtendedTelemetryItem, ITelemetryItem } from '@microsoft/1ds-core-js';
+import * as https from 'https';
+import * as http from 'http';
+import * as os from 'os';
+
+interface SystemInfo {
+ measurements: Record<string, number | undefined>;
+ properties: Record<string, string | boolean | null | undefined>;
+}
+
+export class TelemetryClient extends AppInsightsCore {
+ private readonly systemInfo: SystemInfo = {
+ measurements: {},
+ properties: {},
+ };
+
2024-05-07 07:53:53 +08:00
+ public constructor(
+ private readonly endpoint: string,
+ machineId: string,
+ isContainer: boolean | undefined) {
+ super();
+
+ // os.cpus() can take a very long time sometimes (personally I see 1-2
+ // seconds in a Coder workspace). This adds up significantly, especially
+ // when many telemetry requests are sent during startup, which can cause
+ // connection timeouts. Try to cache as much as we can.
+ try {
+ const cpus = os.cpus();
+ this.systemInfo.measurements.cores = cpus.length;
+ this.systemInfo.properties['common.cpuModel'] = cpus[0].model;
+ } catch (error) {}
+
+ try {
+ this.systemInfo.properties['common.shell'] = os.userInfo().shell;
+ this.systemInfo.properties['common.release'] = os.release();
+ this.systemInfo.properties['common.arch'] = os.arch();
+ } catch (error) {}
+
+ this.systemInfo.properties['common.remoteMachineId'] = machineId;
+ this.systemInfo.properties['common.isContainer'] = isContainer;
+ }
+
+ public override track(item: IExtendedTelemetryItem | ITelemetryItem): void {
+ const options = item.baseData || {}
+ options.measurements = {
+ ...(options.measurements || {}),
+ ...this.systemInfo.measurements,
+ }
+ options.properties = {
+ ...(options.properties || {}),
+ ...this.systemInfo.properties,
+ }
+
+ try {
+ options.measurements.memoryFree = os.freemem();
+ options.measurements.memoryTotal = os.totalmem();
+ } catch (error) {}
+
+ try {
+ const request = (/^http:/.test(this.endpoint) ? http : https).request(this.endpoint, {
+ method: 'POST',
+ headers: {
+ 'Content-Type': 'application/json',
+ },
+ });
+ request.on('error', () => { /* We don't care. */ });
+ request.write(JSON.stringify(options));
+ request.end();
+ } catch (error) {}
+ }
+}
chore: update Code to 1.68 (#5263) * chore: update Code to 1.67 Was able to remove our changes to common/webview.ts since they are upstream now. Other than that no serious changes, just context diffs. * chore: update Code to 1.68 - Upstream moved the web socket endpoint so change the Express route from / to *. That will let web sockets work at any endpoint. - Everything in the workbench config is basically the same but de-indented (upstream extracted it into a separate object which resulted in a de-indent), the ordering is slightly different, and instead of vscodeBase we now need vscodeBase + this._staticRoute since everything is served from a sub-path now. - Move manifest link back to the root since that is where we host our manifest. - Change RemoteAuthoritiesImpl to use the same path building method as in other places (+ instead of using URI.parse/join). - Use existing host/port in RemoteAuthoritiesImpl and BrowserSocketFactory instead of patching them to use window.location (these are set from window.location to begin with so it should be the same result but with less patching). - Since BrowserSocketFactory includes a sub-path now (endpoints were changed upstream to serve from /quality/commit instead of from the root) the patch there has changed to prepend the base to that path (instead of using the base directly). - The workbench HTML now natively supports a base URL in the form of WORKBENCH_WEB_BASE_URL so no need for VS_BASE patches there anymore. - Upstream added type="image/x-icon" so I did as well. - Move the language patch to the end of the series so it is easier to eventually remove. - Remove the existing NLS config in favor of one that supports extensions. - Upstream deleted webview main.js and inlined it into the HTML so move that code (the parent origin check) into both those HTML files (index.html and index-no-csp.html). - The remaining diff is from changes to the surrounding context or a line was changed slightly by upstream (for example renamed files or new arguments like to the remote authority resolver). * fix: modify product.json before building Code injects this into the client during the build process so it needs to be updated before we build. * fix: update inline script nonces * Update HTML base path test * fix: missing commit Code overrides it with nothing. The date is also already injected. * fix: web extensions breaking when the commit changes By just using the marketplace directly instead of going through the backend. I am not sure what the point is when searching extensions already goes directly to the marketplace anyway. But also remove the prefix that breaks this as well because otherwise existing installations will break.
2022-06-22 05:51:46 +08:00
Index: code-server/lib/vscode/src/vs/server/node/webClientServer.ts
===================================================================
--- code-server.orig/lib/vscode/src/vs/server/node/webClientServer.ts
+++ code-server/lib/vscode/src/vs/server/node/webClientServer.ts
@@ -318,6 +318,8 @@ export class WebClientServer {
scope: vscodeBase + '/',
path: base + '/_static/out/browser/serviceWorker.js',
chore: update Code to 1.68 (#5263) * chore: update Code to 1.67 Was able to remove our changes to common/webview.ts since they are upstream now. Other than that no serious changes, just context diffs. * chore: update Code to 1.68 - Upstream moved the web socket endpoint so change the Express route from / to *. That will let web sockets work at any endpoint. - Everything in the workbench config is basically the same but de-indented (upstream extracted it into a separate object which resulted in a de-indent), the ordering is slightly different, and instead of vscodeBase we now need vscodeBase + this._staticRoute since everything is served from a sub-path now. - Move manifest link back to the root since that is where we host our manifest. - Change RemoteAuthoritiesImpl to use the same path building method as in other places (+ instead of using URI.parse/join). - Use existing host/port in RemoteAuthoritiesImpl and BrowserSocketFactory instead of patching them to use window.location (these are set from window.location to begin with so it should be the same result but with less patching). - Since BrowserSocketFactory includes a sub-path now (endpoints were changed upstream to serve from /quality/commit instead of from the root) the patch there has changed to prepend the base to that path (instead of using the base directly). - The workbench HTML now natively supports a base URL in the form of WORKBENCH_WEB_BASE_URL so no need for VS_BASE patches there anymore. - Upstream added type="image/x-icon" so I did as well. - Move the language patch to the end of the series so it is easier to eventually remove. - Remove the existing NLS config in favor of one that supports extensions. - Upstream deleted webview main.js and inlined it into the HTML so move that code (the parent origin check) into both those HTML files (index.html and index-no-csp.html). - The remaining diff is from changes to the surrounding context or a line was changed slightly by upstream (for example renamed files or new arguments like to the remote authority resolver). * fix: modify product.json before building Code injects this into the client during the build process so it needs to be updated before we build. * fix: update inline script nonces * Update HTML base path test * fix: missing commit Code overrides it with nothing. The date is also already injected. * fix: web extensions breaking when the commit changes By just using the marketplace directly instead of going through the backend. I am not sure what the point is when searching extensions already goes directly to the marketplace anyway. But also remove the prefix that breaks this as well because otherwise existing installations will break.
2022-06-22 05:51:46 +08:00
},
+ enableTelemetry: this._productService.enableTelemetry,
+ telemetryEndpoint: this._productService.telemetryEndpoint,
embedderIdentifier: 'server-distro',
extensionsGallery: this._productService.extensionsGallery,
} satisfies Partial<IProductConfiguration>;
Index: code-server/lib/vscode/src/vs/base/common/product.ts
===================================================================
--- code-server.orig/lib/vscode/src/vs/base/common/product.ts
+++ code-server/lib/vscode/src/vs/base/common/product.ts
@@ -64,6 +64,7 @@ export interface IProductConfiguration {
readonly path: string;
readonly scope: string;
}
+ readonly telemetryEndpoint?: string
readonly version: string;
readonly date?: string;
Index: code-server/lib/vscode/src/vs/platform/product/common/product.ts
===================================================================
--- code-server.orig/lib/vscode/src/vs/platform/product/common/product.ts
+++ code-server/lib/vscode/src/vs/platform/product/common/product.ts
@@ -55,7 +55,8 @@ else if (globalThis._VSCODE_PRODUCT_JSON
resourceUrlTemplate: "https://open-vsx.org/vscode/asset/{publisher}/{name}/{version}/Microsoft.VisualStudio.Code.WebResources/{path}",
controlUrl: "",
recommendationsUrl: "",
- })
+ }),
+ telemetryEndpoint: env.CS_TELEMETRY_URL || product.telemetryEndpoint || "https://v1.telemetry.coder.com/track",
});
}