code-server/docs/CONTRIBUTING.md

297 lines
11 KiB
Markdown
Raw Normal View History

<!-- prettier-ignore-start -->
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
2020-02-19 08:07:01 +08:00
# Contributing
2021-09-30 05:59:17 +08:00
- [Requirements](#requirements)
- [Linux-specific requirements](#linux-specific-requirements)
2021-09-30 05:59:17 +08:00
- [Development workflow](#development-workflow)
- [Version updates to Code](#version-updates-to-code)
- [Patching Code](#patching-code)
2021-09-30 05:59:17 +08:00
- [Build](#build)
- [Creating a Standalone Release](#creating-a-standalone-release)
- [Troubleshooting](#troubleshooting)
- [I see "Forbidden access" when I load code-server in the browser](#i-see-forbidden-access-when-i-load-code-server-in-the-browser)
- ["Can only have one anonymous define call per script"](#can-only-have-one-anonymous-define-call-per-script)
- [Help](#help)
2021-09-30 05:59:17 +08:00
- [Test](#test)
- [Unit tests](#unit-tests)
- [Script tests](#script-tests)
- [Integration tests](#integration-tests)
- [End-to-end tests](#end-to-end-tests)
- [Structure](#structure)
- [Modifications to Code](#modifications-to-code)
2021-09-30 05:59:17 +08:00
- [Currently Known Issues](#currently-known-issues)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
<!-- prettier-ignore-end -->
## Requirements
The prerequisites for contributing to code-server are almost the same as those
2024-07-04 05:30:33 +08:00
for [VS Code](https://github.com/Microsoft/vscode/wiki/How-to-Contribute#prerequisites).
Here is what is needed:
- `node` v20.x
- `git` v2.x or greater
- [`git-lfs`](https://git-lfs.github.com)
- [`npm`](https://www.npmjs.com/)
- Used to install JS packages and run scripts
- [`nfpm`](https://nfpm.goreleaser.com/)
- Used to build `.deb` and `.rpm` packages
- [`jq`](https://stedolan.github.io/jq/)
- Used to build code-server releases
- [`gnupg`](https://gnupg.org/index.html)
- All commits must be signed and verified; see GitHub's [Managing commit
signature
verification](https://docs.github.com/en/github/authenticating-to-github/managing-commit-signature-verification)
or follow [this tutorial](https://joeprevite.com/verify-commits-on-github)
- `quilt`
- Used to manage patches to Code
- `rsync` and `unzip`
- Used for code-server releases
2021-07-14 01:22:00 +08:00
- `bats`
- Used to run script unit tests
### Linux-specific requirements
2024-07-04 05:30:33 +08:00
If you're developing code-server on Linux, make sure you have installed or
install the following dependencies:
```shell
sudo apt-get install build-essential g++ libx11-dev libxkbfile-dev libsecret-1-dev libkrb5-dev python-is-python3
```
2024-07-04 05:30:33 +08:00
These are required by Code. See [their Wiki](https://github.com/microsoft/vscode/wiki/How-to-Contribute#prerequisites)
for more information.
## Development workflow
2020-02-19 08:07:01 +08:00
1. `git clone https://github.com/coder/code-server.git` - Clone `code-server`
2. `git submodule update --init` - Clone `vscode` submodule
3. `quilt push -a` - Apply patches to the `vscode` submodule.
4. `npm install` - Install dependencies
5. `npm run watch` - Launch code-server localhost:8080. code-server will be live
reloaded when changes are made; the browser needs to be refreshed manually.
2020-02-19 08:07:01 +08:00
When pulling down changes that include modifications to the patches you will
need to apply them with `quilt`. If you pull down changes that update the
`vscode` submodule you will need to run `git submodule update --init` and
re-apply the patches.
2020-12-18 05:08:51 +08:00
2024-07-04 05:30:33 +08:00
When you make a change that affects people deploying the marketplace please
update the changelog as part of your PR.
2024-07-04 05:30:33 +08:00
Note that building code-server takes a very, very long time, and loading it in
the browser in development mode also takes a very, very long time.
2024-07-04 05:30:33 +08:00
Display language (Spanish, etc) support only works in a full build; it will not
work in development mode.
2024-07-04 05:30:33 +08:00
Generally we prefer that PRs be squashed into `main` but you can rebase or merge
if it is important to keep the individual commits (make sure to clean up the
commits first if you are doing this).
### Version updates to Code
2024-07-04 05:30:33 +08:00
1. Remove any patches with `quilt pop -a`.
2. Update the `lib/vscode` submodule to the desired upstream version branch.
1. `cd lib/vscode && git checkout release/1.66 && cd ../..`
2. `git add lib && git commit -m "chore: update to Code <version>"`
3. Apply the patches one at a time (`quilt push`). If the application succeeds
but the lines changed, update the patch with `quilt refresh`. If there are
conflicts, then force apply with `quilt push -f`, manually add back the
rejected code, then run `quilt refresh`.
4. From the code-server **project root**, run `npm install`.
2024-07-04 05:30:33 +08:00
5. Check the Node.js version that's used by Electron (which is shipped with VS
Code. If necessary, update our version of Node.js to match.
### Patching Code
2024-07-04 05:30:33 +08:00
1. You can go through the patch stack with `quilt push` and `quilt pop`.
2. Create a new patch (`quilt new {name}.diff`) or use an existing patch.
3. Add the file(s) you are patching (`quilt add [-P patch] {file}`). A file
**must** be added before you make changes to it.
2024-07-04 05:30:33 +08:00
4. Make your changes. Patches do not need to be independent of each other but
each patch must result in a working code-server without any broken in-between
states otherwise they are difficult to test and modify.
2024-07-04 05:30:33 +08:00
5. Add your changes to the patch (`quilt refresh`)
6. Add a comment in the patch about the reason for the patch and how to
reproduce the behavior it fixes or adds. Every patch should have an e2e test
as well.
2020-12-18 05:08:51 +08:00
### Build
2020-02-19 08:07:01 +08:00
2024-07-04 05:30:33 +08:00
You can build a full production as follows:
```shell
2024-07-04 05:30:33 +08:00
git submodule update --init
quilt push -a
npm install
npm run build
VERSION=0.0.0 npm run build:vscode
npm run release
```
2024-07-04 05:30:33 +08:00
This does not keep `node_modules`. If you want them to be kept, use
`KEEP_MODULES=1 npm run release`
chore: upgrade Code to 1.66 (#5135) * chore: upgrade Code to 1.66 * docs: update docs for Code upgrades * fixup!: docs * chore: update vscode submodule * chore: update integration patch * chore: update node-version patch * chore: update github-auth patch They completely changed how auth is handled for GitHub in https://github.com/microsoft/vscode/pull/145424 so our patch may not work. Will need to test and revisit. * refactor: remove postinstall patch It appears they renamed postinstall.js to postinstall.mjs and removed the use of `rimraf` which means our patch is no longer needed! :tada: https://github.com/microsoft/vscode/commit/b0e8554cced292871a67748a18926cfd02f4e840 * chore: refresh local-storage patch * chore: refresh service-worker patch * chore: bulk refresh patches * fixup!: docs formatting * refactor: remove unused last-opened patch * fixup!: formatting docs * fixup!: formatting docs * refactor: remove rsync postinstall * Revert "refactor: remove rsync postinstall" This reverts commit 8d6b613e9d779ba18d0297710614516cde108bcf. * refactor: update postinstall.js to .mjs * feat(patches): add parent-origin bypass * docs(patches): add notes for testing store-socket * docs(patches): update testing info for node-version * refactor(patches): delete github-auth.diff patch * docs(patches): add notes for testing connection-type * fixup!: delete github-auth patch * fixup!: update connection type testing * docs(patches): add notes to insecure-notification.diff * docs(patches): add nots for update-check.diff * fixup!: remove comma in integration patch * fix(e2e): disable workspace trust * refactor: add --no-default-rc for yarn install * feat(patches): remove yarnrc in presinstall * fixup!: silly mistake * docs: add note about KEEP_MODULES=1 * docs(patches): add testing notes for node-version * refactor(patches): remove node-version It appears this is no longer needed due to the `remote/package.json` now which targets node rather than electron. * fixup!: add cd ../.. to code upgrade instructions * fixup!: add note to yarn --production flag * fixup!: make parent-origin easier to upstream * Revert "refactor(patches): delete github-auth.diff patch" This reverts commit 31a354a34345309fadc475491b392d7601e51a32. * Revert "fixup!: delete github-auth patch" This reverts commit bdeb5212e8c7be6cadd109941b486a4bcdae69fa. * Merge webview origin patch into webview patch * Remove unused post-install patch * Prevent builtin extensions from updating * Refresh sourcemaps patch * Update Node to v16 This matches the version in ./lib/vscode/remote/.yarnrc. I changed the engine to exactly 16 since if you use any different version it will just not work since the modules will have been built for 16 (due to the .yarnrc). * Replace fs.rmdir with fs.rm Node is showing a deprecation warning about it. * Update github-auth patch The local credentials provider is no longer used when there is a remote so this code moved into the backend web credential provider. * Prevent fs.rm from erroring about non-existent files We were using fs.rmdir which presumably did not have the same behavior in v14 (in v16 fs.rmdir also errors). * Install Python 3 in CentOS CI container Co-authored-by: Asher <ash@coder.com>
2022-05-05 05:58:49 +08:00
Run your build:
```shell
cd release
npm install --omit=dev # Skip if you used KEEP_MODULES=1
# Runs the built JavaScript with Node.
node .
```
2024-07-04 05:30:33 +08:00
Then, to build the release package:
```shell
npm run release:standalone
npm run test:integration
npm run package
```
> On Linux, the currently running distro will become the minimum supported
2024-07-04 05:30:33 +08:00
> version. In our GitHub Actions CI, we use CentOS 8 for maximum compatibility.
> If you need your builds to support older distros, run the build commands
> inside a Docker container with all the build requirements installed.
2020-10-08 01:26:57 +08:00
#### Creating a Standalone Release
Part of the build process involves creating standalone releases. At the time of
writing, we do this for the following platforms/architectures:
- Linux amd64 (.tar.gz, .deb, and .rpm)
- Linux arm64 (.tar.gz, .deb, and .rpm)
- Linux arm7l (.tar.gz)
- Linux armhf.deb
- Linux armhf.rpm
2024-07-04 05:30:33 +08:00
- macOS arm64.tar.gz
Currently, these are compiled in CI using the `npm run release:standalone`
command in the `release.yaml` workflow. We then upload them to the draft release
and distribute via GitHub Releases.
### Troubleshooting
#### I see "Forbidden access" when I load code-server in the browser
2024-07-04 05:30:33 +08:00
This means your patches didn't apply correctly. We have a patch to remove the
auth from vanilla Code because we use our own.
2024-07-04 05:30:33 +08:00
Try popping off the patches with `quilt pop -a` and reapplying with `quilt push
-a`.
#### "Can only have one anonymous define call per script"
2024-07-04 05:30:33 +08:00
Code might be trying to use a dev or prod HTML in the wrong context. You can try
re-running code-server and setting `VSCODE_DEV=1`.
### Help
2024-07-04 05:30:33 +08:00
If you get stuck or need help, you can always start a new GitHub Discussion
[here](https://github.com/coder/code-server/discussions). One of the maintainers
will respond and help you out.
## Test
2021-06-18 02:38:04 +08:00
There are four kinds of tests in code-server:
2021-06-18 02:38:04 +08:00
1. Unit tests
2. Script tests
3. Integration tests
4. End-to-end tests
2021-06-18 02:38:04 +08:00
### Unit tests
2021-06-18 02:38:04 +08:00
Our unit tests are written in TypeScript and run using
[Jest](https://jestjs.io/), the testing framework].
2021-06-18 02:38:04 +08:00
These live under [test/unit](../test/unit).
2024-07-04 05:30:33 +08:00
We use unit tests for functions and things that can be tested in isolation. The
file structure is modeled closely after `/src` so it's easy for people to know
where test files should live.
2021-06-18 02:38:04 +08:00
### Script tests
Our script tests are written in bash and run using [bats](https://github.com/bats-core/bats-core).
These tests live under `test/scripts`.
2024-07-04 05:30:33 +08:00
We use these to test anything related to our scripts (most of which live under
`ci`).
### Integration tests
2021-06-18 02:38:04 +08:00
These are a work in progress. We build code-server and run tests with `npm run
2024-07-04 05:30:33 +08:00
test:integration`, which ensures that code-server builds work on their
respective platforms.
2021-06-18 02:38:04 +08:00
Our integration tests look at components that rely on one another. For example,
testing the CLI requires us to build and package code-server.
2021-06-18 02:38:04 +08:00
### End-to-end tests
2021-06-18 02:38:04 +08:00
The end-to-end (e2e) tests are written in TypeScript and run using
[Playwright](https://playwright.dev/).
2021-06-18 02:38:04 +08:00
These live under [test/e2e](../test/e2e).
Before the e2e tests run, we run `globalSetup`, which eliminates the need to log
in before each test by preserving the authentication state.
2021-06-18 02:38:04 +08:00
Take a look at `codeServer.test.ts` to see how you would use it (see
`test.use`).
2021-06-18 02:38:04 +08:00
We also have a model where you can create helpers to use within tests. See
[models/CodeServer.ts](../test/e2e/models/CodeServer.ts) for an example.
2021-06-18 02:38:04 +08:00
2020-05-18 04:53:08 +08:00
## Structure
2024-07-04 05:30:33 +08:00
code-server essentially serves as an HTTP API for logging in and starting a
remote Code process.
2020-05-18 04:53:08 +08:00
The CLI code is in [src/node](../src/node) and the HTTP routes are implemented
in [src/node/routes](../src/node/routes).
2020-05-18 04:53:08 +08:00
Most of the meaty parts are in the Code portion of the codebase under
[lib/vscode](../lib/vscode), which we describe next.
2020-05-18 04:53:08 +08:00
### Modifications to Code
Our modifications to Code can be found in the [patches](../patches) directory.
We pull in Code as a submodule pointing to an upstream release branch.
In v1 of code-server, we had Code as a submodule and used a single massive patch
that split the codebase into a front-end and a server. The front-end consisted
of the UI code, while the server ran the extensions and exposed an API to the
front-end for file access and all UI needs.
Over time, Microsoft added support to Code to run it on the web. They had made
the front-end open source, but not the server. As such, code-server v2 (and
later) uses the Code front-end and implements the server. We did this by using a
Git subtree to fork and modify Code.
Microsoft eventually made the server open source and we were able to reduce our
changes significantly. Some time later we moved back to a submodule and patches
(managed by `quilt` this time instead of the mega-patch).
As the web portion of Code continues to mature, we'll be able to shrink and
possibly eliminate our patches. In the meantime, upgrading the Code version
requires us to ensure that our changes are still applied correctly and work as
intended. In the future, we'd like to run Code unit tests against our builds to
ensure that features work as expected.
2020-05-18 04:53:08 +08:00
> We have [extension docs](../ci/README.md) on the CI and build system.
2020-05-18 04:53:08 +08:00
If the functionality you're working on does NOT depend on code from Code, please
move it out and into code-server.
2020-05-18 04:53:08 +08:00
### Currently Known Issues
2020-05-18 04:53:08 +08:00
- Creating custom Code extensions and debugging them doesn't work
- Extension profiling and tips are currently disabled