diff --git a/AGENTS.md b/AGENTS.md index 43892ab..4ac959d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -5,13 +5,13 @@ saved time, add it here. ## Project -Online Mancala (the Turkish *Mangala* rules) with a web app, an Android/iOS app and a Node backend. +Online Mancala (the Turkish *Mangala* rules) with a web app, an Android/iOS app and a Java (Quarkus) backend. | Path | What | |---|---| -| `packages/mancala.js` | Game engine; rules are plug-ins in `src/common/game_rules` | +| `packages/mancala.js` | Game engine used by the apps; rules are plug-ins in `src/common/game_rules`. The backend has a Java port | | `packages/core` | Code shared by the apps: models, WebSocket client (`RTMTWS`), channel names, history helpers, themes, `PitAnimator`, translations (en, tr) | -| `apps/backend` | Express + `ws`: matchmaking, games, auth, saved games, HTTP API | +| `apps/backend` | Java 25 + Quarkus: matchmaking, games, auth, saved games, HTTP API, the Java port of the engine | | `apps/frontend` | React web app, built with Parcel | | `apps/mobile` | React Native app | | `apps/keycloak` | Keycloak image with the realm, the login theme and a Java extension for guest login | @@ -22,15 +22,15 @@ Online Mancala (the Turkish *Mangala* rules) with a web app, an Android/iOS app | `TODO.md` | Open work, known bugs and deployment steps; read it before starting | Specs that exist: `keycloak-auth` (login, guests, tokens), `game-history` (saved games, live games, replay), -`game-clock` (5 minutes per player, ending games on time) and `web-redesign` (the web app's look: themes, board, -pages). +`game-clock` (5 minutes per player, ending games on time), `web-redesign` (the web app's look: themes, board, +pages) and `quarkus-backend` (the backend's move from Node to Java and Quarkus). ## Running locally ```sh yarn build:packages # repo root; builds mancala.js and core cd apps && docker compose up -d --build postgres valkey keycloak # needs apps/.env, see README.md -cd apps/backend && yarn dev # http://localhost:5005 +cd apps/backend && ./mvnw quarkus:dev # http://localhost:5005, needs a JDK 25 cd apps/frontend && yarn dev # http://localhost:1234 (must be 1234, Keycloak only allows that) ``` @@ -42,9 +42,9 @@ unless `useLocalServer` is set in `apps/mobile/src/const/config.ts`. | What | Command | Notes | |---|---|---| -| Engine | `cd packages/mancala.js && npx jest` | | +| Engine | `cd packages/mancala.js && npx jest` | After changing the rules, change the Java port too and record the parity games again (`apps/backend/README.md`) | | Core | `cd packages/core && npx jest` | | -| Backend | `cd apps/backend && npx tsc --noEmit -p . && npx jest` | Needs Docker: Postgres and Valkey run in testcontainers | +| Backend | `cd apps/backend && ./mvnw test` | Needs a JDK 25 and Docker: Postgres and Valkey run with Quarkus Dev Services | | Keycloak extension | `cd apps/keycloak/extension && mvn test` | Needs Docker; starts Keycloak with the real realm file | | Web app | `cd apps/frontend && yarn build` | There is no `tsconfig.json`, so nothing type-checks it; use a temporary one when needed (with `"types": ["styled-jsx", "react", "react-dom"]`). 1 type error (`react-dom/client`) is known | | Mobile | `cd apps/mobile && npx tsc --noEmit -p .` and `npx react-native bundle --platform android --dev false --entry-file index.js --bundle-output /tmp/x.bundle` | 11 type errors are known and older than recent work | @@ -66,25 +66,38 @@ There is no automated end-to-end suite. Changes to the game flow were checked by ## Things that are easy to get wrong **Packages and builds** -- The apps use `packages/core` and `packages/mancala.js` through `link:` dependencies, i.e. their built `dist`. - After changing a package, run `yarn build` in it (or `yarn dev` to rebuild on change), or the apps use stale code. - Running dev servers do not notice a rebuilt package: restart the backend, and restart Parcel after deleting - `apps/frontend/.parcel-cache`. +- The web and mobile apps use `packages/core` and `packages/mancala.js` through `link:` dependencies, i.e. their built + `dist`. After changing a package, run `yarn build` in it (or `yarn dev` to rebuild on change), or the apps use stale + code. + Running dev servers do not notice a rebuilt package: restart Parcel after deleting `apps/frontend/.parcel-cache`. + The backend does not use the packages. - Backend and frontend Docker images are built from the repo root: `docker build -f apps/backend/Dockerfile .`. - The mobile app uses `RTMTWS` from core (its own `src/rtmt` copy is unused), so changing the core WebSocket client changes mobile too. Metro needs `watchFolders` and `nodeModulesPaths` (already in `metro.config.js`). **Backend** +- The game rules exist twice: `packages/mancala.js` for the clients and `app.playmancala.engine` in the backend. + Change both; `EngineParityTest` replays games recorded with the TypeScript engine and fails if they differ. +- All live state (games, matchmaking, sockets, previews) belongs to the game loop thread (`GameLoop`). Touch it only + in tasks running there; never block the loop on Postgres, Valkey or HTTP. Other threads use `GameLoop.call`. +- The JSON must stay what the Node backend sent and stored: TypeScript leaves out `undefined` but keeps `null`, so only + optional fields get `@JsonInclude(NON_NULL)` (e.g. `result: null` stays). Valkey snapshots and keys, the tables and + `schema_migrations` are shared with the Node backend's format; tests check data the Node backend wrote. - User ids: Keycloak users are `kc:`; legacy mobile users are their `userKey`, which is a secret. Anything - saved or listed publicly goes through `publicUserId()`; clients get `toClientGame()`, never spectator ids. + saved or listed publicly goes through `PublicIds.publicUserId`; clients get `toClientGame()`, never spectator ids. - Live games: memory is the source of truth, Valkey is a write-through copy restored on startup; the backend assumes a single instance. Finished games are written to Postgres once, with retry when it is down. - Postgres or Valkey being down must not stop games; reads fall back, writes are retried. - The game clock is stored as each player's time at the start of the turn plus `turnStartedAt`; compute the time - left with `getClockTimes`, never store a ticking value. `GAME_CLOCK_MS=40000` makes games short for manual tests. -- `RTMT.listenOnClientConnectionChange` adds a listener (several are allowed). -- Jest 27 lacks Node's web globals that testcontainers needs; `tests/support/NodeEnvironment.js` copies them in. -- A Valkey client pointed at a dead port keeps Jest alive for `disconnectTimeout`; tests pass a short one. + left with `GameClocks.getClockTimes`, never store a ticking value. `GAME_CLOCK_MS=40000` makes games short for + manual tests. +- `Rtmt.listenOnClientConnectionChange` adds a listener (several are allowed). +- Unit tests build the logic with fakes (`FakeRtmt`, in-memory stores, `ManualScheduler` instead of timers and + direct executors); only storage, HTTP and WebSocket tests are `@QuarkusTest`s. `FakeTokenVerifier` replaces + Keycloak in all `@QuarkusTest`s. +- Quarkus dev mode's debugger is moved to port 5006 (`pom.xml`), because 5005 is the backend's port. +- The Maven wrapper checks the `.zip` it downloads; without `unzip` it fetches a `.tar.gz` and fails the check (the + Dockerfile installs `unzip`). **Web app** - styled-jsx only scopes JSX written directly in a component's render; elements built in helper functions or diff --git a/README.md b/README.md index 6d1c657..4716a9a 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,8 @@ Mancala is an open source application for playing mancala game online. ## Development -The apps use the local `packages/mancala.js` and `packages/core` through `link:` dependencies, so build those first: +The web and mobile apps use the local `packages/mancala.js` and `packages/core` through `link:` dependencies, so build +those first: ```sh yarn build:packages # from the repo root @@ -26,8 +27,13 @@ EOF docker compose up -d --build postgres valkey keycloak ``` -Postgres is on `localhost:5433` and Valkey on `localhost:6379`; the backend's defaults point there. Then run -`yarn dev` in `apps/backend` (port 5005) and in `apps/frontend` (port 1234). +Postgres is on `localhost:5433` and Valkey on `localhost:6379`; the backend's defaults point there. Then run the +backend (Java 25 and Quarkus, see `apps/backend/README.md`) and the web app: + +```sh +cd apps/backend && ./mvnw quarkus:dev # port 5005 +cd apps/frontend && yarn dev # port 1234 +``` Docker images for the backend and frontend are built from the repo root: diff --git a/TODO.md b/TODO.md index 21efd86..7b73634 100644 --- a/TODO.md +++ b/TODO.md @@ -69,6 +69,10 @@ See `docs/specs/web-redesign`. - [ ] Deploy Keycloak (`apps/.env` with `KEYCLOAK_DB_PASSWORD`, `KEYCLOAK_ADMIN_PASSWORD`, `KEYCLOAK_HOSTNAME=https://auth.playmancala.app`) and reload nginx with the new `auth.playmancala.app` block - [ ] Deploy the backend before the web app; the new web app needs token auth on the server +- [ ] The backend is now Java (Quarkus, see `docs/specs/quarkus-backend`); the compose file passes the database as + `DATABASE_JDBC_URL`/`DATABASE_USERNAME`/`DATABASE_PASSWORD` built from the same `MANCALA_DB_PASSWORD`, so + `apps/.env` needs no change. It reads the Node backend's database and live games, so it can replace it (and be + rolled back) without ending games - [ ] Release the mobile update with the game clock before or with the backend: older app versions do not show the clock and do not handle `on_game_end`, so a game they lose on time does not end for them - [ ] Back up the Postgres volume (Keycloak and saved games), e.g. a daily `pg_dumpall` @@ -83,7 +87,8 @@ See `docs/specs/web-redesign`. `~/.gradle/gradle.properties` or environment variables, and consider resetting the upload key - [ ] Add `apps/mobile/mancala-upload-key.jks` to `.gitignore` (only `android/app/mancala-upload-key.jks` is ignored) - [ ] Validate moves before running them instead of treating every invalid move as a crash: each one writes a - file to `crashes/` and is broadcast to the players (`GameManager.onGameError`) + file to `crashes/` and is broadcast to the players (`GameManager.onGameError`, kept as it was in the Node + backend) ## Game rules @@ -95,7 +100,8 @@ See `docs/specs/web-redesign`. ## Bugs - [ ] `PitAnimator.dispose` removes a newly bound function, so the theme listener is never removed -- [ ] `onGameError` broadcasts an `Error`, which serializes to `{}` +- [ ] `onGameError` broadcasts `{}` instead of the error (the Java backend sends `{}` on purpose, as the Node backend + did) ## Maintenance @@ -103,6 +109,8 @@ See `docs/specs/web-redesign`. - [ ] Mixed lockfiles: `packages/*` and `apps/mobile` have both `yarn.lock` and `package-lock.json` - [ ] Align TypeScript and Jest versions across packages - [ ] Tests for `MatchMaker`; `PitAnimator` is only tested for the first frame of a move -- [ ] CI that runs the tests +- [ ] CI that runs the tests (the backend's need a JDK 25 and Docker) +- [ ] The rules exist twice, in `packages/mancala.js` and in the backend's Java port; a parity test keeps them alike, + but every rule change has to be made in both - [ ] Pre-existing type errors: 4 in the frontend, 11 in mobile; the frontend has no `tsconfig.json`, so nothing type-checks it diff --git a/apps/backend/README.md b/apps/backend/README.md new file mode 100644 index 0000000..fb5c3ef --- /dev/null +++ b/apps/backend/README.md @@ -0,0 +1,104 @@ +# Mancala Backend + +The game server: matchmaking, live games over a WebSocket, saved games and a small HTTP API. Java 25 and Quarkus 3.40, +built with Maven. The design and the reasons for it are in `docs/specs/quarkus-backend`. + +## Getting started + +You need a JDK 25 (e.g. Temurin); Maven comes with the wrapper. Postgres, Valkey and Keycloak run with Docker Compose, +see the README in the repo root. + +```sh +./mvnw quarkus:dev # http://localhost:5005; a debugger can attach on 5006 +./mvnw test # needs Docker: Postgres and Valkey run with Quarkus Dev Services +./mvnw package # target/quarkus-app/quarkus-run.jar +java -jar target/quarkus-app/quarkus-run.jar +``` + +The Docker image is built from the repo root: `docker build -f apps/backend/Dockerfile .`. It listens on port 6000. + +## Code + +| Package | What | +|---|---| +| `engine` | The game rules, a port of `packages/mancala.js` | +| `model` | The JSON sent to clients and stored, and the shared `ObjectMapper` | +| `loop` | `GameLoop`, the single thread that owns all live game state | +| `auth` | Keycloak token verification | +| `rtmt` | The WebSocket protocol (`RtmtServer`) and the Quarkus endpoint | +| `game` | Games, matchmaking, the clock, live previews, snapshots and saving finished games | +| `history` | Saved games in Postgres, migrations, the lists for the API | +| `live` | Valkey: live game snapshots, games waiting to be saved, the cache | +| `http` | The HTTP API | +| `Backend` | Wiring and startup | + +All live state (games, matchmaking, sockets) is only touched on the game loop, like on the Node backend's event loop. +Blocking work never runs there: Postgres writes run on the recorder's thread, Valkey writes on their own ordered +thread, token checks on virtual threads, HTTP requests on Quarkus' workers (which read live games through +`GameLoop.call`). + +The clients use the TypeScript engine, the server the Java one; they must play exactly alike. +`EngineParityTest` replays games recorded with the TypeScript engine. After changing the rules in both, record them +again: + +```sh +yarn --cwd ../../packages/mancala.js build && node scripts/record-engine-games.js +``` + +## Authentication + +Clients log in with Keycloak (see `apps/keycloak`), either as a guest or with an account, and send the access token +over the WebSocket right after connecting: + +```json +{ "channel": "auth", "message": { "token": "" } } +``` + +The server answers on `on_auth` with the user (`{ id, name, isGuest }`) or on `on_auth_error` and closes the socket +with code 4001. Other messages are ignored until the socket is authenticated, and connections that do not +authenticate within 5 seconds are closed. Send `auth` again with the new token after refreshing it. +User ids of Keycloak users are the token subject prefixed with `kc:`. + +| Variable | Default | Purpose | +|---|---|---| +| `KEYCLOAK_ISSUER` | `http://localhost:8080/realms/mancala` | Token issuer; must match the `iss` claim exactly | +| `KEYCLOAK_JWKS_URL` | issuer + `/protocol/openid-connect/certs` | Where to fetch signing keys, if the backend reaches Keycloak at a different address | +| `KEYCLOAK_CLIENT_IDS` | `mancala-web` | Comma-separated clients whose tokens are accepted | +| `LEGACY_USERKEY_AUTH` | `true` | Also accept old clients that connect with `?userKey=` and the `/register` endpoint. Turn off once no released client uses it, because a userKey is not a secret the server can check | +| `PORT` | `5005` (`6000` in the image) | HTTP and WebSocket port | + +## Saved games and live games + +Finished games are saved in Postgres; live games are copied to Valkey after every change and restored when the +backend starts, so a restart or deploy does not end them. The design is in `docs/specs/game-history`. + +| Variable | Default | Purpose | +|---|---|---| +| `DATABASE_JDBC_URL` | `jdbc:postgresql://localhost:5433/mancala` | Saved games; migrations run on startup | +| `DATABASE_USERNAME`, `DATABASE_PASSWORD` | `mancala`, `mancala` | | +| `VALKEY_URL` | `redis://localhost:6379` | Live game state and short-lived caches | + +The backend keeps running when either is down: games are still played, finished games are saved once Postgres is +back, and lists are served without the missing data. The tables, the `schema_migrations` table and the Valkey keys +are the Node backend's, so the two can replace each other in either direction. + +## Game clock + +Each player has a clock for the whole game; only the clock of the player whose turn it is runs, also while they are +offline. A move that arrives after the mover's time has run out is refused, and `GameClockScheduler` ends games whose +running clock reaches zero once a second, so the opponent wins on time even if nobody is connected. Time while the +backend is down is not charged. The design is in `docs/specs/game-clock`. + +| Variable | Default | Purpose | +|---|---|---| +| `GAME_CLOCK_MS` | `300000` | Time each player has for a game; a short value such as `40000` helps when testing by hand | + +| Route | Auth | Response | +|---|---|---| +| `GET /game/:id` | – | A live game, or `{ status: "finished", record }` | +| `GET /games/popular?limit=5` | – | `{ kind: "live" \| "finished", games }` | +| `GET /games/me/active` | Bearer token | `{ game }` or `{ game: null }` | +| `GET /games/me?limit=3` | Bearer token | `{ games }`, newest first | + +Requests are logged to `access.log`, and a move that fails writes the game to `crashes/`, both in the working +directory. diff --git a/docs/specs/quarkus-backend/design.md b/docs/specs/quarkus-backend/design.md index bd84774..65e7dbe 100644 --- a/docs/specs/quarkus-backend/design.md +++ b/docs/specs/quarkus-backend/design.md @@ -38,13 +38,13 @@ apps/backend/ GameClock, Seat, GameResult, EndReason, GameEndMessage, MoveRecord, GameRecord, FinishedGameSummary, LiveGameSummary, ClientGame, Channels; Json (the shared ObjectMapper) loop/ GameLoop (the game thread), Scheduler (what the logic needs from it), Cancellable - auth/ AuthConfig, TokenVerifier, KeycloakTokenVerifier, InvalidTokenException + auth/ TokenVerifier, KeycloakTokenVerifier (made from config by TokenVerifierProducer), InvalidTokenException rtmt/ Rtmt (interface used by the game logic), RtmtServer, RtmtSocket, RtmtEndpoint (Quarkus) game/ BackendGame, GameStore, MatchMaker, GameManager, GameClocks, GameClockScheduler, GameSerialization (LiveGameSnapshot), GameRecorder, LiveGames, PublicIds, CrashLog history/ GameHistoryRepository, PostgresGameHistoryRepository, Migrations, GameHistoryService live/ LiveGameStore, ValkeyLiveGameStore, Cache, ValkeyCache - http/ GameResource, RootResource, BearerAuth, exception mapper + http/ GameResource (every route), BearerAuth, ErrorMapper (503 for failures) Backend.java wiring and startup (a @Startup bean) ``` diff --git a/docs/specs/quarkus-backend/requirements.md b/docs/specs/quarkus-backend/requirements.md index a163408..49879e0 100644 --- a/docs/specs/quarkus-backend/requirements.md +++ b/docs/specs/quarkus-backend/requirements.md @@ -1,6 +1,6 @@ # Quarkus backend: requirements -Status: implemented on branch `feature/quarkus-backend`. +Status: implemented on branch `feature/quarkus-backend`; checked in Chrome against the local stack. Scope: `apps/backend`, its Docker image, `apps/compose.yaml` and the docs. ## Goals diff --git a/docs/specs/quarkus-backend/tasks.md b/docs/specs/quarkus-backend/tasks.md index fcce883..1994bf3 100644 --- a/docs/specs/quarkus-backend/tasks.md +++ b/docs/specs/quarkus-backend/tasks.md @@ -16,5 +16,5 @@ Work through the tasks in order; each ends with its tests passing and a commit. wiring and startup, configuration; `@QuarkusTest`s for the API and the WebSocket. - [x] 7. **Replace the Node backend**: move the project to `apps/backend`, Dockerfile, `.dockerignore`, compose, remove the Node sources; build the image. -- [ ] 8. **End to end and docs**: the Chrome tests from `design.md` against the local stack; `README.md`, +- [x] 8. **End to end and docs**: the Chrome tests from `design.md` against the local stack; `README.md`, `apps/backend/README.md`, `AGENTS.md`, `TODO.md`.