From 0903f614559bf0de3320697e8edaa05db2146848 Mon Sep 17 00:00:00 2001 From: Halit Aksoy Date: Fri, 16 Oct 2026 11:53:13 +0300 Subject: [PATCH] docs: describe the Java backend The backend README, how to run and test it, what is easy to get wrong with the game loop and the shared data formats, and the deployment note. The web app was played against it in Chrome: a full game with a spectator, a reload, a restart mid-game, the replay and a game lost on time. --- AGENTS.md | 47 ++++++---- README.md | 12 ++- TODO.md | 14 ++- apps/backend/README.md | 104 +++++++++++++++++++++ docs/specs/quarkus-backend/design.md | 4 +- docs/specs/quarkus-backend/requirements.md | 2 +- docs/specs/quarkus-backend/tasks.md | 2 +- 7 files changed, 158 insertions(+), 27 deletions(-) create mode 100644 apps/backend/README.md diff --git a/AGENTS.md b/AGENTS.md index 43892ab..4ac959d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -5,13 +5,13 @@ saved time, add it here. ## Project -Online Mancala (the Turkish *Mangala* rules) with a web app, an Android/iOS app and a Node backend. +Online Mancala (the Turkish *Mangala* rules) with a web app, an Android/iOS app and a Java (Quarkus) backend. | Path | What | |---|---| -| `packages/mancala.js` | Game engine; rules are plug-ins in `src/common/game_rules` | +| `packages/mancala.js` | Game engine used by the apps; rules are plug-ins in `src/common/game_rules`. The backend has a Java port | | `packages/core` | Code shared by the apps: models, WebSocket client (`RTMTWS`), channel names, history helpers, themes, `PitAnimator`, translations (en, tr) | -| `apps/backend` | Express + `ws`: matchmaking, games, auth, saved games, HTTP API | +| `apps/backend` | Java 25 + Quarkus: matchmaking, games, auth, saved games, HTTP API, the Java port of the engine | | `apps/frontend` | React web app, built with Parcel | | `apps/mobile` | React Native app | | `apps/keycloak` | Keycloak image with the realm, the login theme and a Java extension for guest login | @@ -22,15 +22,15 @@ Online Mancala (the Turkish *Mangala* rules) with a web app, an Android/iOS app | `TODO.md` | Open work, known bugs and deployment steps; read it before starting | Specs that exist: `keycloak-auth` (login, guests, tokens), `game-history` (saved games, live games, replay), -`game-clock` (5 minutes per player, ending games on time) and `web-redesign` (the web app's look: themes, board, -pages). +`game-clock` (5 minutes per player, ending games on time), `web-redesign` (the web app's look: themes, board, +pages) and `quarkus-backend` (the backend's move from Node to Java and Quarkus). ## Running locally ```sh yarn build:packages # repo root; builds mancala.js and core cd apps && docker compose up -d --build postgres valkey keycloak # needs apps/.env, see README.md -cd apps/backend && yarn dev # http://localhost:5005 +cd apps/backend && ./mvnw quarkus:dev # http://localhost:5005, needs a JDK 25 cd apps/frontend && yarn dev # http://localhost:1234 (must be 1234, Keycloak only allows that) ``` @@ -42,9 +42,9 @@ unless `useLocalServer` is set in `apps/mobile/src/const/config.ts`. | What | Command | Notes | |---|---|---| -| Engine | `cd packages/mancala.js && npx jest` | | +| Engine | `cd packages/mancala.js && npx jest` | After changing the rules, change the Java port too and record the parity games again (`apps/backend/README.md`) | | Core | `cd packages/core && npx jest` | | -| Backend | `cd apps/backend && npx tsc --noEmit -p . && npx jest` | Needs Docker: Postgres and Valkey run in testcontainers | +| Backend | `cd apps/backend && ./mvnw test` | Needs a JDK 25 and Docker: Postgres and Valkey run with Quarkus Dev Services | | Keycloak extension | `cd apps/keycloak/extension && mvn test` | Needs Docker; starts Keycloak with the real realm file | | Web app | `cd apps/frontend && yarn build` | There is no `tsconfig.json`, so nothing type-checks it; use a temporary one when needed (with `"types": ["styled-jsx", "react", "react-dom"]`). 1 type error (`react-dom/client`) is known | | Mobile | `cd apps/mobile && npx tsc --noEmit -p .` and `npx react-native bundle --platform android --dev false --entry-file index.js --bundle-output /tmp/x.bundle` | 11 type errors are known and older than recent work | @@ -66,25 +66,38 @@ There is no automated end-to-end suite. Changes to the game flow were checked by ## Things that are easy to get wrong **Packages and builds** -- The apps use `packages/core` and `packages/mancala.js` through `link:` dependencies, i.e. their built `dist`. - After changing a package, run `yarn build` in it (or `yarn dev` to rebuild on change), or the apps use stale code. - Running dev servers do not notice a rebuilt package: restart the backend, and restart Parcel after deleting - `apps/frontend/.parcel-cache`. +- The web and mobile apps use `packages/core` and `packages/mancala.js` through `link:` dependencies, i.e. their built + `dist`. After changing a package, run `yarn build` in it (or `yarn dev` to rebuild on change), or the apps use stale + code. + Running dev servers do not notice a rebuilt package: restart Parcel after deleting `apps/frontend/.parcel-cache`. + The backend does not use the packages. - Backend and frontend Docker images are built from the repo root: `docker build -f apps/backend/Dockerfile .`. - The mobile app uses `RTMTWS` from core (its own `src/rtmt` copy is unused), so changing the core WebSocket client changes mobile too. Metro needs `watchFolders` and `nodeModulesPaths` (already in `metro.config.js`). **Backend** +- The game rules exist twice: `packages/mancala.js` for the clients and `app.playmancala.engine` in the backend. + Change both; `EngineParityTest` replays games recorded with the TypeScript engine and fails if they differ. +- All live state (games, matchmaking, sockets, previews) belongs to the game loop thread (`GameLoop`). Touch it only + in tasks running there; never block the loop on Postgres, Valkey or HTTP. Other threads use `GameLoop.call`. +- The JSON must stay what the Node backend sent and stored: TypeScript leaves out `undefined` but keeps `null`, so only + optional fields get `@JsonInclude(NON_NULL)` (e.g. `result: null` stays). Valkey snapshots and keys, the tables and + `schema_migrations` are shared with the Node backend's format; tests check data the Node backend wrote. - User ids: Keycloak users are `kc:`; legacy mobile users are their `userKey`, which is a secret. Anything - saved or listed publicly goes through `publicUserId()`; clients get `toClientGame()`, never spectator ids. + saved or listed publicly goes through `PublicIds.publicUserId`; clients get `toClientGame()`, never spectator ids. - Live games: memory is the source of truth, Valkey is a write-through copy restored on startup; the backend assumes a single instance. Finished games are written to Postgres once, with retry when it is down. - Postgres or Valkey being down must not stop games; reads fall back, writes are retried. - The game clock is stored as each player's time at the start of the turn plus `turnStartedAt`; compute the time - left with `getClockTimes`, never store a ticking value. `GAME_CLOCK_MS=40000` makes games short for manual tests. -- `RTMT.listenOnClientConnectionChange` adds a listener (several are allowed). -- Jest 27 lacks Node's web globals that testcontainers needs; `tests/support/NodeEnvironment.js` copies them in. -- A Valkey client pointed at a dead port keeps Jest alive for `disconnectTimeout`; tests pass a short one. + left with `GameClocks.getClockTimes`, never store a ticking value. `GAME_CLOCK_MS=40000` makes games short for + manual tests. +- `Rtmt.listenOnClientConnectionChange` adds a listener (several are allowed). +- Unit tests build the logic with fakes (`FakeRtmt`, in-memory stores, `ManualScheduler` instead of timers and + direct executors); only storage, HTTP and WebSocket tests are `@QuarkusTest`s. `FakeTokenVerifier` replaces + Keycloak in all `@QuarkusTest`s. +- Quarkus dev mode's debugger is moved to port 5006 (`pom.xml`), because 5005 is the backend's port. +- The Maven wrapper checks the `.zip` it downloads; without `unzip` it fetches a `.tar.gz` and fails the check (the + Dockerfile installs `unzip`). **Web app** - styled-jsx only scopes JSX written directly in a component's render; elements built in helper functions or diff --git a/README.md b/README.md index 6d1c657..4716a9a 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,8 @@ Mancala is an open source application for playing mancala game online. ## Development -The apps use the local `packages/mancala.js` and `packages/core` through `link:` dependencies, so build those first: +The web and mobile apps use the local `packages/mancala.js` and `packages/core` through `link:` dependencies, so build +those first: ```sh yarn build:packages # from the repo root @@ -26,8 +27,13 @@ EOF docker compose up -d --build postgres valkey keycloak ``` -Postgres is on `localhost:5433` and Valkey on `localhost:6379`; the backend's defaults point there. Then run -`yarn dev` in `apps/backend` (port 5005) and in `apps/frontend` (port 1234). +Postgres is on `localhost:5433` and Valkey on `localhost:6379`; the backend's defaults point there. Then run the +backend (Java 25 and Quarkus, see `apps/backend/README.md`) and the web app: + +```sh +cd apps/backend && ./mvnw quarkus:dev # port 5005 +cd apps/frontend && yarn dev # port 1234 +``` Docker images for the backend and frontend are built from the repo root: diff --git a/TODO.md b/TODO.md index 21efd86..7b73634 100644 --- a/TODO.md +++ b/TODO.md @@ -69,6 +69,10 @@ See `docs/specs/web-redesign`. - [ ] Deploy Keycloak (`apps/.env` with `KEYCLOAK_DB_PASSWORD`, `KEYCLOAK_ADMIN_PASSWORD`, `KEYCLOAK_HOSTNAME=https://auth.playmancala.app`) and reload nginx with the new `auth.playmancala.app` block - [ ] Deploy the backend before the web app; the new web app needs token auth on the server +- [ ] The backend is now Java (Quarkus, see `docs/specs/quarkus-backend`); the compose file passes the database as + `DATABASE_JDBC_URL`/`DATABASE_USERNAME`/`DATABASE_PASSWORD` built from the same `MANCALA_DB_PASSWORD`, so + `apps/.env` needs no change. It reads the Node backend's database and live games, so it can replace it (and be + rolled back) without ending games - [ ] Release the mobile update with the game clock before or with the backend: older app versions do not show the clock and do not handle `on_game_end`, so a game they lose on time does not end for them - [ ] Back up the Postgres volume (Keycloak and saved games), e.g. a daily `pg_dumpall` @@ -83,7 +87,8 @@ See `docs/specs/web-redesign`. `~/.gradle/gradle.properties` or environment variables, and consider resetting the upload key - [ ] Add `apps/mobile/mancala-upload-key.jks` to `.gitignore` (only `android/app/mancala-upload-key.jks` is ignored) - [ ] Validate moves before running them instead of treating every invalid move as a crash: each one writes a - file to `crashes/` and is broadcast to the players (`GameManager.onGameError`) + file to `crashes/` and is broadcast to the players (`GameManager.onGameError`, kept as it was in the Node + backend) ## Game rules @@ -95,7 +100,8 @@ See `docs/specs/web-redesign`. ## Bugs - [ ] `PitAnimator.dispose` removes a newly bound function, so the theme listener is never removed -- [ ] `onGameError` broadcasts an `Error`, which serializes to `{}` +- [ ] `onGameError` broadcasts `{}` instead of the error (the Java backend sends `{}` on purpose, as the Node backend + did) ## Maintenance @@ -103,6 +109,8 @@ See `docs/specs/web-redesign`. - [ ] Mixed lockfiles: `packages/*` and `apps/mobile` have both `yarn.lock` and `package-lock.json` - [ ] Align TypeScript and Jest versions across packages - [ ] Tests for `MatchMaker`; `PitAnimator` is only tested for the first frame of a move -- [ ] CI that runs the tests +- [ ] CI that runs the tests (the backend's need a JDK 25 and Docker) +- [ ] The rules exist twice, in `packages/mancala.js` and in the backend's Java port; a parity test keeps them alike, + but every rule change has to be made in both - [ ] Pre-existing type errors: 4 in the frontend, 11 in mobile; the frontend has no `tsconfig.json`, so nothing type-checks it diff --git a/apps/backend/README.md b/apps/backend/README.md new file mode 100644 index 0000000..fb5c3ef --- /dev/null +++ b/apps/backend/README.md @@ -0,0 +1,104 @@ +# Mancala Backend + +The game server: matchmaking, live games over a WebSocket, saved games and a small HTTP API. Java 25 and Quarkus 3.40, +built with Maven. The design and the reasons for it are in `docs/specs/quarkus-backend`. + +## Getting started + +You need a JDK 25 (e.g. Temurin); Maven comes with the wrapper. Postgres, Valkey and Keycloak run with Docker Compose, +see the README in the repo root. + +```sh +./mvnw quarkus:dev # http://localhost:5005; a debugger can attach on 5006 +./mvnw test # needs Docker: Postgres and Valkey run with Quarkus Dev Services +./mvnw package # target/quarkus-app/quarkus-run.jar +java -jar target/quarkus-app/quarkus-run.jar +``` + +The Docker image is built from the repo root: `docker build -f apps/backend/Dockerfile .`. It listens on port 6000. + +## Code + +| Package | What | +|---|---| +| `engine` | The game rules, a port of `packages/mancala.js` | +| `model` | The JSON sent to clients and stored, and the shared `ObjectMapper` | +| `loop` | `GameLoop`, the single thread that owns all live game state | +| `auth` | Keycloak token verification | +| `rtmt` | The WebSocket protocol (`RtmtServer`) and the Quarkus endpoint | +| `game` | Games, matchmaking, the clock, live previews, snapshots and saving finished games | +| `history` | Saved games in Postgres, migrations, the lists for the API | +| `live` | Valkey: live game snapshots, games waiting to be saved, the cache | +| `http` | The HTTP API | +| `Backend` | Wiring and startup | + +All live state (games, matchmaking, sockets) is only touched on the game loop, like on the Node backend's event loop. +Blocking work never runs there: Postgres writes run on the recorder's thread, Valkey writes on their own ordered +thread, token checks on virtual threads, HTTP requests on Quarkus' workers (which read live games through +`GameLoop.call`). + +The clients use the TypeScript engine, the server the Java one; they must play exactly alike. +`EngineParityTest` replays games recorded with the TypeScript engine. After changing the rules in both, record them +again: + +```sh +yarn --cwd ../../packages/mancala.js build && node scripts/record-engine-games.js +``` + +## Authentication + +Clients log in with Keycloak (see `apps/keycloak`), either as a guest or with an account, and send the access token +over the WebSocket right after connecting: + +```json +{ "channel": "auth", "message": { "token": "" } } +``` + +The server answers on `on_auth` with the user (`{ id, name, isGuest }`) or on `on_auth_error` and closes the socket +with code 4001. Other messages are ignored until the socket is authenticated, and connections that do not +authenticate within 5 seconds are closed. Send `auth` again with the new token after refreshing it. +User ids of Keycloak users are the token subject prefixed with `kc:`. + +| Variable | Default | Purpose | +|---|---|---| +| `KEYCLOAK_ISSUER` | `http://localhost:8080/realms/mancala` | Token issuer; must match the `iss` claim exactly | +| `KEYCLOAK_JWKS_URL` | issuer + `/protocol/openid-connect/certs` | Where to fetch signing keys, if the backend reaches Keycloak at a different address | +| `KEYCLOAK_CLIENT_IDS` | `mancala-web` | Comma-separated clients whose tokens are accepted | +| `LEGACY_USERKEY_AUTH` | `true` | Also accept old clients that connect with `?userKey=` and the `/register` endpoint. Turn off once no released client uses it, because a userKey is not a secret the server can check | +| `PORT` | `5005` (`6000` in the image) | HTTP and WebSocket port | + +## Saved games and live games + +Finished games are saved in Postgres; live games are copied to Valkey after every change and restored when the +backend starts, so a restart or deploy does not end them. The design is in `docs/specs/game-history`. + +| Variable | Default | Purpose | +|---|---|---| +| `DATABASE_JDBC_URL` | `jdbc:postgresql://localhost:5433/mancala` | Saved games; migrations run on startup | +| `DATABASE_USERNAME`, `DATABASE_PASSWORD` | `mancala`, `mancala` | | +| `VALKEY_URL` | `redis://localhost:6379` | Live game state and short-lived caches | + +The backend keeps running when either is down: games are still played, finished games are saved once Postgres is +back, and lists are served without the missing data. The tables, the `schema_migrations` table and the Valkey keys +are the Node backend's, so the two can replace each other in either direction. + +## Game clock + +Each player has a clock for the whole game; only the clock of the player whose turn it is runs, also while they are +offline. A move that arrives after the mover's time has run out is refused, and `GameClockScheduler` ends games whose +running clock reaches zero once a second, so the opponent wins on time even if nobody is connected. Time while the +backend is down is not charged. The design is in `docs/specs/game-clock`. + +| Variable | Default | Purpose | +|---|---|---| +| `GAME_CLOCK_MS` | `300000` | Time each player has for a game; a short value such as `40000` helps when testing by hand | + +| Route | Auth | Response | +|---|---|---| +| `GET /game/:id` | – | A live game, or `{ status: "finished", record }` | +| `GET /games/popular?limit=5` | – | `{ kind: "live" \| "finished", games }` | +| `GET /games/me/active` | Bearer token | `{ game }` or `{ game: null }` | +| `GET /games/me?limit=3` | Bearer token | `{ games }`, newest first | + +Requests are logged to `access.log`, and a move that fails writes the game to `crashes/`, both in the working +directory. diff --git a/docs/specs/quarkus-backend/design.md b/docs/specs/quarkus-backend/design.md index bd84774..65e7dbe 100644 --- a/docs/specs/quarkus-backend/design.md +++ b/docs/specs/quarkus-backend/design.md @@ -38,13 +38,13 @@ apps/backend/ GameClock, Seat, GameResult, EndReason, GameEndMessage, MoveRecord, GameRecord, FinishedGameSummary, LiveGameSummary, ClientGame, Channels; Json (the shared ObjectMapper) loop/ GameLoop (the game thread), Scheduler (what the logic needs from it), Cancellable - auth/ AuthConfig, TokenVerifier, KeycloakTokenVerifier, InvalidTokenException + auth/ TokenVerifier, KeycloakTokenVerifier (made from config by TokenVerifierProducer), InvalidTokenException rtmt/ Rtmt (interface used by the game logic), RtmtServer, RtmtSocket, RtmtEndpoint (Quarkus) game/ BackendGame, GameStore, MatchMaker, GameManager, GameClocks, GameClockScheduler, GameSerialization (LiveGameSnapshot), GameRecorder, LiveGames, PublicIds, CrashLog history/ GameHistoryRepository, PostgresGameHistoryRepository, Migrations, GameHistoryService live/ LiveGameStore, ValkeyLiveGameStore, Cache, ValkeyCache - http/ GameResource, RootResource, BearerAuth, exception mapper + http/ GameResource (every route), BearerAuth, ErrorMapper (503 for failures) Backend.java wiring and startup (a @Startup bean) ``` diff --git a/docs/specs/quarkus-backend/requirements.md b/docs/specs/quarkus-backend/requirements.md index a163408..49879e0 100644 --- a/docs/specs/quarkus-backend/requirements.md +++ b/docs/specs/quarkus-backend/requirements.md @@ -1,6 +1,6 @@ # Quarkus backend: requirements -Status: implemented on branch `feature/quarkus-backend`. +Status: implemented on branch `feature/quarkus-backend`; checked in Chrome against the local stack. Scope: `apps/backend`, its Docker image, `apps/compose.yaml` and the docs. ## Goals diff --git a/docs/specs/quarkus-backend/tasks.md b/docs/specs/quarkus-backend/tasks.md index fcce883..1994bf3 100644 --- a/docs/specs/quarkus-backend/tasks.md +++ b/docs/specs/quarkus-backend/tasks.md @@ -16,5 +16,5 @@ Work through the tasks in order; each ends with its tests passing and a commit. wiring and startup, configuration; `@QuarkusTest`s for the API and the WebSocket. - [x] 7. **Replace the Node backend**: move the project to `apps/backend`, Dockerfile, `.dockerignore`, compose, remove the Node sources; build the image. -- [ ] 8. **End to end and docs**: the Chrome tests from `design.md` against the local stack; `README.md`, +- [x] 8. **End to end and docs**: the Chrome tests from `design.md` against the local stack; `README.md`, `apps/backend/README.md`, `AGENTS.md`, `TODO.md`.