diff --git a/.gitignore b/.gitignore index d45de5ca..8719d1d4 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,6 @@ .DS_Store .idea/ +.env engine/bin/ /db-lab-run/ @@ -13,3 +14,5 @@ engine/bin/ /engine/configs/ci_checker.yml engine/meta + +ui/packages/shared/dist/ diff --git a/.gitlab/agents/k8s-cluster-1/config.yaml b/.gitlab/agents/k8s-cluster-1/config.yaml new file mode 100644 index 00000000..73481f44 --- /dev/null +++ b/.gitlab/agents/k8s-cluster-1/config.yaml @@ -0,0 +1,3 @@ +ci_access: + projects: + - id: postgres-ai/database-lab diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..a4267581 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,23 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Build/Test/Lint Commands +- Build all components: `cd engine && make build` +- Lint code: `cd engine && make lint` +- Run unit tests: `cd engine && make test` +- Run integration tests: `cd engine && make test-ci-integration` +- Run a specific test: `cd engine && GO111MODULE=on go test -v ./path/to/package -run TestName` +- Run UI: `cd ui && pnpm start:ce` (Community Edition) or `pnpm start:platform` + +## Code Style Guidelines +- Go code follows "Effective Go" and "Go Code Review Comments" guidelines +- Use present tense and imperative mood in commit messages +- Limit first commit line to 72 characters +- All Git commits must be signed +- Format Go code with `cd engine && make fmt` +- Use error handling with pkg/errors +- Follow standard Go import ordering +- Group similar functions together +- Error messages should be descriptive and actionable +- UI uses pnpm for package management \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f32b4abf..4d399f35 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -23,11 +23,11 @@ These are mostly guidelines, not rules. Use your best judgment, and feel free to - [Git commit messages](#git-commit-messages) - [Go styleguide](#go-styleguide) - [Documentation styleguide](#documentation-styleguide) + - [API design and testing](#api-design-and-testing) + - [UI development](#ui-development) - [Development setup](#development-setup) - [Repo overview](#repo-overview) - --- @@ -121,6 +121,45 @@ We encourage you to follow the principles described in the following documents: - [Effective Go](https://go.dev/doc/effective_go) - [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments) +### Message style guide +Consistent messaging is important throughout the codebase. Follow these guidelines for errors, logs, and user-facing messages: + +#### Error messages +- Lowercase for internal errors and logs: `failed to start session` (no ending period) +- Uppercase for user-facing errors: `Requested object does not exist. Specify your request.` (with ending period) +- Omit articles ("a", "an", "the") for brevity: use `failed to update clone` not `failed to update the clone` +- Be specific and actionable whenever possible +- For variable interpolation, use consistent formatting: `failed to find clone: %s` + +#### CLI output +- Use concise, action-oriented language +- Present tense with ellipsis for in-progress actions: `Creating clone...` + - Ellipsis (`...`) indicates an ongoing process where the user should wait + - Always follow up with a completion message when the operation finishes +- Past tense with period for results: `Clone created successfully.` +- Include relevant identifiers (IDs, names) in output + +#### Progress indication +- Use ellipsis (`...`) to indicate that an operation is in progress and the user should wait +- For longer operations, consider providing percentage or step indicators: `Cloning database... (25%)` +- When an operation with ellipsis completes, always provide a completion message without ellipsis +- Example sequence: + ``` + Creating clone... + Clone "test-clone" created successfully. + ``` + +#### UI messages +- Be consistent with terminology across UI and documentation +- For confirmations, use format: `{Resource} {action} successfully.` +- For errors, provide clear next steps when possible +- Use sentence case for all messages (capitalize first word only) + +#### Commit messages +- Start with lowercase type prefix: `fix:`, `feat:`, `docs:`, etc. +- Use imperative mood: `add feature` not `added feature` +- Provide context in the body if needed + ### Documentation styleguide Documentation for Database Lab Engine and additional components is hosted at https://postgres.ai/docs and is maintained in this GitLab repo: https://gitlab.com/postgres-ai/docs. @@ -132,6 +171,94 @@ We're building documentation following the principles described at https://docum Learn more: https://documentation.divio.com/. +### API design and testing +The DBLab API follows RESTful principles with these key guidelines: +- Clear resource-based URL structure +- Consistent usage of HTTP methods (GET, POST, DELETE, etc.) +- Standardized error responses +- Authentication via API tokens +- JSON for request and response bodies +- Comprehensive documentation with examples + +#### API Documentation +We use readme.io to host the API docs: https://dblab.readme.io/ and https://api.dblab.dev. + +When updating the API specification: +1. Make changes to the OpenAPI spec file in `engine/api/swagger-spec/` +2. Upload it to readme.io as a new documentation version +3. Review and publish the new version + +#### Testing with Postman and Newman +Postman collection is generated based on the OpenAPI spec file, using [Portman](https://github.com/apideck-libraries/portman). + +##### Setup and Generation +1. Install Portman: `npm install -g @apideck/portman` +2. Generate Postman collection file: + ``` + portman --cliOptionsFile engine/api/postman/portman-cli.json + ``` + +##### Test Structure Best Practices +- Arrange tests in logical flows (create, read, update, delete) +- Use environment variables to store and pass data between requests +- For object creation tests, capture the ID in the response to use in subsequent requests +- Add validation tests for response status, body structure, and expected values +- Clean up created resources at the end of test flows + +##### CI/CD Integration +The Postman collection is automatically run in CI/CD pipelines using Newman. For local testing: +``` +newman run engine/api/postman/dblab_api.postman_collection.json -e engine/api/postman/branching.aws.postgres.ai.postman_environment.json +``` + +### UI development +The Database Lab Engine UI contains two main packages: +- `@postgres.ai/platform` - Platform version of UI +- `@postgres.ai/ce` - Community Edition version of UI +- `@postgres.ai/shared` - Common modules shared between packages + +#### Working with UI packages +At the repository root: +- `pnpm install` - Install all dependencies +- `npm run build -ws` - Build all packages +- `npm run start -w @postgres.ai/platform` - Run Platform UI in dev mode +- `npm run start -w @postgres.ai/ce` - Run Community Edition UI in dev mode + +_Note: Don't use commands for `@postgres.ai/shared` - it's a dependent package that can't be run or built directly_ + +#### Platform UI Development +1. Set up environment variables: + ```bash + cd ui/packages/platform + cp .env_example_dev .env + ``` +2. Edit `.env` to set: + - `REACT_APP_API_URL_PREFIX` to point to dev API server + - `REACT_APP_TOKEN_DEBUG` to set your JWT token +3. Start development server: `pnpm run start` + +#### CI pipelines for UI code +To deploy UI changes, tag the commit with `ui/` prefix and push it: +```shell +git tag ui/1.0.12 +git push origin ui/1.0.12 +``` + +#### Handling Vulnerabilities +When addressing vulnerabilities in UI packages: +1. Update the affected package to a newer version if available +2. For sub-package vulnerabilities, try using [npm-force-resolutions](https://www.npmjs.com/package/npm-force-resolutions) +3. As a last resort, consider forking the package locally + +For code-related issues: +1. Consider rewriting JavaScript code in TypeScript +2. Follow recommendations from security analysis tools +3. Only ignore false positives when absolutely necessary + +#### TypeScript Migration +- `@postgres.ai/shared` and `@postgres.ai/ce` are written in TypeScript +- `@postgres.ai/platform` is partially written in TypeScript with ongoing migration efforts + ### Repo overview The [postgres-ai/database-lab](https://gitlab.com/postgres-ai/database-lab) repo contains 2 components: - [Database Lab Engine](https://gitlab.com/postgres-ai/database-lab/-/tree/master/engine) @@ -140,7 +267,6 @@ The [postgres-ai/database-lab](https://gitlab.com/postgres-ai/database-lab) repo - [Database Lab CLI](https://gitlab.com/postgres-ai/database-lab/-/tree/master/engine/cmd/cli) - [Database Lab UI](https://gitlab.com/postgres-ai/database-lab/-/tree/master/ui) - [Community Edition](https://gitlab.com/postgres-ai/database-lab/-/tree/master/ui/packages/ce) - - [Platform](https://gitlab.com/postgres-ai/database-lab/-/tree/master/ui/packages/platform) - [Shared components](https://gitlab.com/postgres-ai/database-lab/-/tree/master/ui/packages/shared) Components have a separate version, denoted by either: @@ -191,10 +317,27 @@ Components have a separate version, denoted by either: ### Building from source -Use `Makefile` to build Database Lab components from source. +The Database Lab Engine provides multiple build targets in its `Makefile`: + +```bash +cd engine +make help # View all available build targets +make build # Build all components (Server, CLI, CI Checker) +make build-dle # Build Database Lab Engine binary and Docker image +make test # Run unit tests +``` + +You can also build specific components: + +```bash +# Build the CLI for all supported platforms +make build-client + +# Build the Server in debug mode +make build-debug -Run `make help` to see all available targets. +# Build and run DLE locally +make run-dle +``` - +See our [GitLab Container Registry](https://gitlab.com/postgres-ai/database-lab/container_registry) to find pre-built images for development branches. diff --git a/README.md b/README.md index 73b8d499..9eada025 100644 --- a/README.md +++ b/README.md @@ -8,18 +8,18 @@