diff --git a/docs/specs/game-bot/design.md b/docs/specs/game-bot/design.md new file mode 100644 index 0000000..79fd051 --- /dev/null +++ b/docs/specs/game-bot/design.md @@ -0,0 +1,210 @@ +# Game bot: design + +See `requirements.md` for what is built; requirement ids (R1.1, …) are referenced below. Builds on `game-clock` +(the clock), `game-history` (saved games, live previews) and `quarkus-backend` (the Java backend and its engine). + +## Decisions + +| Decision | Reason | +|---|---| +| The bot runs in the backend, in Java | The rules are already in Java (`app.playmancala.engine`) and kept equal to the TypeScript engine by `EngineParityTest`. A bot game is then an ordinary server game: the clock, saving, spectators, previews and restarts work without changes in the clients (R4). A bot on the device would only add offline play, at the cost of a second bot in TypeScript (out of scope). | +| A compact board, `Position` (`int[14]` and whose turn it is), in the engine package | Searching means playing millions of moves; `MancalaGame` builds `Pit` objects, lists and animation steps for each, which is far too slow. `Position` plays a move into an existing array without allocating. It is in `engine`, not `bot`, so other code can use it too, e.g. checking a move before running it (see `TODO.md`, Security). It is tested against `MancalaGame` with the recorded parity games and random games. | +| `MancalaGame` is not rebuilt on `Position` now | `MancalaGame` also produces the game steps clients animate, the history and the JSON the Node backend wrote, and it must stay equal to the TypeScript engine. Rebuilding it is a separate refactor once `Position` has proven itself (out of scope). | +| Negamax with alpha-beta pruning, limited by depth and a node budget, not by time | Mangala is small (6 moves at most per turn), so alpha-beta is strong enough: at depth 12 the bot plays close to perfectly. Limits that do not depend on time make a move depend only on the position, the strength and the seed, so a game can be reproduced for debugging; a slower machine only takes longer. | +| Strength (0–100) sets three things: search depth, how much the bot prefers the best move (softmax temperature) and how often it plays a random move | Depth alone gives few, coarse steps and a predictable bot. With softmax, a weaker bot more often plays a move that is nearly as good, which looks like a human mistake; the random moves make Easy beatable by beginners (R2.3). Interpolating a table gives smooth steps between levels. | +| Each level is a range of strengths; the strength is drawn per game and varies by ±5 per move | R2.2. The ranges have gaps between them so neighbouring levels stay clearly apart (R2.4). | +| Randomness comes from a per-game seed and the move number | The bot's choice of a move is reproducible from the saved game, and a restart does not change how it plays (R4.6). The seed and strength are kept with the live game and the saved game. | +| Searching runs on its own small thread pool; the result goes back to the game loop, which checks that the game is still waiting for that move | The game loop must never wait (R3.5, `AGENTS.md`). The pool has a fixed number of threads, so many bot games share the CPU instead of starving the loop and the WebSocket threads. A result is dropped if the game ended or moved on meanwhile. | +| The bot's waiting time is computed on the server: the previous move's animation (300 ms per step, as `PitAnimator`) plus a random 1–3 seconds | R3.2. Clients do not need to know about bots for this. Searching happens during the wait, so the time it takes is not added to it. | +| Bot ids are `bot:`, e.g. `bot:medium` | One stable id per level reads well in saved games and lets later statistics group games by level. It is public and cannot belong to a person: Keycloak ids start with `kc:` and legacy ids must be UUIDs (`RtmtServer`). Because one id plays many games at once, `GameStore` does not map it to a game. Clients recognise bots by the id, so `PlayerSummary` keeps its shape. | +| The offer to play the bot (R1.1) is a timer in the client | The server does not need to know how long someone has waited. Matchmaking stays as it is; choosing a level sends a new message that also ends the search on the server. | +| Bot games are ordinary games everywhere: clock, saving, spectators, live previews, popular games | R4. Fewer special cases. The bot's moves go through the same code as a player's moves (R3.1). | + +## Shared types (`packages/core`) + +```ts +// rtmt/channel_names.ts +export const channel_new_bot_game = "new_bot_game"; // client -> server: { level: BotLevel } + +// models/Bot.ts +export type BotLevel = "easy" | "medium" | "hard" | "master"; +export const BOT_LEVELS: BotLevel[] = ["easy", "medium", "hard", "master"]; +export const BOT_OFFER_DELAY_MS = 10_000; // R1.1 +export function isBotId(userId: string | undefined): boolean; // starts with "bot:" +export function getBotLevel(userId: string | undefined): BotLevel | undefined; +``` + +- `getPlayerDisplayName(player, t)` takes `userId` too (every caller has it) and returns e.g. "Bot · Orta" for + bots (R4.1). +- Texts (en, tr): `PlayWithBot` "Play with bot" / "Bota karşı oyna", `NoOpponentYet` "No opponent yet. Play with + the bot?" / "Henüz rakip bulunamadı. Bota karşı oynamak ister misin?", `Bot` "Bot" / "Bot", `BotLevelEasy`, + `BotLevelMedium`, `BotLevelHard`, `BotLevelMaster` "Easy", "Medium", "Hard", "Master" / "Kolay", "Orta", "Zor", + "Usta", `PlayAgain` "Play again" / "Tekrar oyna". + +## Backend + +### Modules +``` +engine/Position.java compact board and the rules on it +bot/BotLevel.java the levels, their wire names and strength ranges +bot/BotSettings.java record (level, strength, seed); kept with a bot game +bot/BotStrength.java strength -> depth, temperature, random move chance +bot/BotSearch.java negamax with alpha-beta; scores of the root moves +bot/BotPolicy.java chooses a move: strength, search, softmax, random moves +bot/BotPlayer.java plays the bot's turns in live games +db/migrations/003_bot.sql bot column +``` +`engine` and `bot` (except `BotPlayer`) have no Quarkus or game loop code, so they can be tested and run on their +own. + +### `Position` +```java +public final class Position { + // CommonBoard layout: player 1's pits 0-5, bank 6; player 2's pits 7-12, bank 13. + public static Position initial(boolean player1ToMove); + public static Position of(int[] stones, boolean player1ToMove); + public static Position of(MancalaGame game); + public int[] stones(); // a copy + public boolean isPlayer1ToMove(); + public int legalPits(int[] into); // pit numbers 0-5 of the mover's non-empty pits; returns the count + // Plays pit 0-5 of the mover into `into` (may be this). Returns true if the mover plays again. + public boolean play(int pit, Position into); + public boolean isOver(); // one side is empty + public int bankDifference(); // mover's bank minus the opponent's +} +``` +`play` applies the rules in the engine's order: sowing (one stone goes to the next pit, otherwise the first stone +goes back into the played pit; the opponent's bank is skipped), last stone in an own empty pit, even count in an +opponent's pit, last stone in the own bank (extra turn) or the turn passes, and clearing the board when a side is +empty. + +### Search (`BotSearch`) +- Negamax with alpha-beta. A move that gives an extra turn keeps the side to move, so its child's score is not + negated. +- Scores are from the mover's view, in hundredths of a stone. An ended game scores + `±100_000 + 100 × final bank difference`; a game where a bank holds more than 24 stones is decided and scores as + ended. Other leaves: `100 × bank difference` plus small terms for stones on each side and moves that give an extra + turn; the weights are tuned in task 3. +- Move order: extra turns, then captures, then the rest, then the best move of the previous iteration. +- Iterative deepening up to the depth; each root move is searched with a full window so every root move gets an + exact score (the softmax needs them). If the node budget (3 million) runs out, the scores of the last completed + depth are used. + +### Strength (`BotLevel`, `BotStrength`) + +| Level | Wire name | Strength | +|---|---|---| +| Easy | `easy` | 5–25 | +| Medium | `medium` | 30–55 | +| Hard | `hard` | 60–80 | +| Master | `master` | 85–100 | + +Starting values, interpolated linearly between rows (depth rounded); tuned in task 3 until R2.4 holds: + +| Strength | Depth | Temperature (stones) | Random move chance | +|---|---|---|---| +| 0 | 1 | 4 | 35% | +| 25 | 2 | 2 | 15% | +| 50 | 4 | 1 | 5% | +| 75 | 7 | 0.4 | 1% | +| 100 | 12 | 0.05 | 0% | + +### Choosing a move (`BotPolicy`) +```java +static int chooseMove(Position position, BotSettings bot, int moveNumber); // pit 0-5 +``` +1. A random source from `bot.seed()` and `moveNumber` (number of moves played in the game). +2. This move's strength: `bot.strength()` ±5, kept within 0–100. +3. With the random move chance: a random legal pit. +4. Otherwise search to the depth and pick a root move with probability ∝ `exp(score / (temperature × 100))`. +5. With a single legal pit, play it without searching. + +### Bot games in the game logic (`GameManager`) +- New channel `new_bot_game` with `{ level }`. An unknown level is logged and ignored. A player with a game in + progress gets `on_new_game_rejected` (R1.5). Otherwise `matchMaker.cancel(userId)`, then a game with the player + as player 1 and `bot:` as player 2, a random first mover (R1.6), and + `bot = new BotSettings(level, random strength in the level's range, random seed)`; then `startGame` as usual. +- `BackendGame` gains `BotSettings bot` (null for games between people) and `botId()`. +- The bot's `PlayerSummary` is `(bot:, "Bot", false)`. +- `onGameMove` is split: the socket handler finds the game and calls `playMove(game, playerId, pitIndex)`, which + holds today's logic (clock check, engine, clock charge, updates, ending). The bot calls the same method (R3.1). +- `onTurnStarted(Consumer)`: called after a game starts, after a move that does not end the game + (including an extra turn) and for each restored game. +- Connection info shows the bot online (R3.4); messages to the bot's id are not sent. +- `GameStore.setGameByUser` and `removeGameByPlayer` ignore bot ids. +- `PublicIds.publicUserId` returns bot ids as they are. + +### `BotPlayer` +```java +public BotPlayer(GameManager gameManager, GameStore gameStore, GameLoop loop, Executor searchExecutor) +``` +Listens to `onTurnStarted`. When it is the bot's turn and no move is being worked out for that game: +1. On the loop: copy the position (`Position.of`), note the move number and compute + `readyAt = now + animationMs(last move) + thinkMs(level)`. `animationMs` is 300 ms per `GAME_STEP_GAME_MOVE` step + of the last move plus 300 ms. `thinkMs` is random: Easy 0.8–1.8 s, Medium 1.0–2.2 s, Hard 1.2–2.6 s, Master + 1.4–3.0 s. With less than 30 s on the bot's clock it is 0.3 s (R3.3). +2. On `searchExecutor`: `BotPolicy.chooseMove`. If it throws, the error is logged and the first legal pit is used, + so the game never waits for a bot that failed. +3. Back on the loop (`loop.execute`, then `loop.schedule` for what is left until `readyAt`): if the game is still + live, it is the same `BackendGame`, the move number has not changed and it is the bot's turn, call + `gameManager.playMove(game, botId, pit)`; otherwise drop the result. + +`searchExecutor` is a fixed pool of `BOT_SEARCH_THREADS` platform threads (default: half the processors, at least +1), created and closed in `Backend`. Unit tests pass a direct executor and `ManualScheduler`. + +### Restore (R4.6) +- `LiveGameSnapshot` gains `bot` (`@JsonInclude(NON_NULL)`); the snapshot stays version 2, since the field is + optional. `fromSnapshot` restores it. A snapshot with a bot player but no `bot` field (not expected) gets the + middle strength of the level and a new seed. +- `restoreLiveGames` calls the turn listeners, so a bot whose turn it was moves. + +### Saving (R4.3) +- `GameRecord` gains `bot` (`@JsonInclude(NON_NULL)`): `{ level, strength, seed }`, sent with `GET /game/:id`; + clients do not use it. It allows tuning the levels from real games later. +- Migration `003_bot`: `ALTER TABLE games ADD COLUMN bot jsonb;` (NULL for games between people). + `PostgresGameHistoryRepository` writes and reads it. + +### Configuration +`BOT_SEARCH_THREADS` (default: half the processors, at least 1). + +## Web app +- `components/BotLevelPicker.tsx`: the four levels as buttons; the last chosen level is preselected and saved in + local storage (R1.4). +- Home page: a "Play with bot" button next to "New Game" opens the picker in a `FloatingPanel`; choosing a level + goes to `/loby?bot=` (R1.3). +- `LobyPage`: with `?bot=`, it sends `new_bot_game` instead of `new_game`. Otherwise, after + `BOT_OFFER_DELAY_MS` without a game, it shows `NoOpponentYet` and the picker below the search; choosing a level + sends `new_bot_game` (R1.1, R1.2). The existing `on_game_start` and `on_new_game_rejected` handling stays. +- `GamePage` / `GameOverCard`: in a bot game, the main button is "Play again" and goes to `/loby?bot=` with + the bot's level (R4.4). +- Bot names: every place that shows a player uses `getPlayerDisplayName` with the player's `userId` (R4.1). + +## Mobile app +- `LobyScreen`: the same offer after `BOT_OFFER_DELAY_MS`, and a `botLevel` route parameter that sends + `new_bot_game` at once. +- `HomeScreen`: a "Play with bot" button with the level choice; the last level is kept in MMKV. +- Bot names through `getPlayerDisplayName`. + +## Testing +- `Position`: replays the recorded parity games (`/engine/recorded-games.json`) and a few thousand random games + move by move against `MancalaGame`, comparing stones, whose turn it is and the end. +- `BotSearch`: finds an extra turn, a capture and a win in one or two moves; the same position, depth and budget + give the same scores; the node budget is respected. +- `BotStrength` and `BotLevel`: interpolation, the ranges, strengths drawn stay in range. +- `BotPolicy`: the same seed and move number give the same move; random moves only at the expected rate; a + single legal pit is played; R2.5 with a fixed opponent. +- Calibration: `BotTournamentTest`, tagged `calibration` and left out of `./mvnw test` + (`./mvnw test -Dgroups=calibration`), plays 200 games for each pair of neighbouring levels and Master against + Easy, alternating the first mover, and fails if R2.4 does not hold. It also reports the average search time per + level. A quick version (20 games, Master against Easy) runs with the normal tests. +- `BotPlayer` with `FakeRtmt`, `ManualScheduler` and a direct executor: the bot moves after the delay; extra turns; + a game that ended or moved on while searching drops the result; a failing search plays the first legal pit; + low time shortens the wait; a restored game where it is the bot's turn gets a move. +- `GameManager`: `new_bot_game` starts a game with the bot and ends the search; it is refused during a game; + unknown levels are ignored; the bot shows as online and gets no messages; the saved record keeps the bot id and + `bot`; leaving and running out of time end the game as usual. +- Postgres repository: the `bot` column is saved and read. Snapshots with and without `bot` restore. +- End to end in Chrome with `GAME_CLOCK_MS=60000`: the offer appears after 10 s in the lobby and starts a game; + "Play with bot" on the home page; the bot moves after the animation; the game ends and shows in recent games and + the replay as "Bot · "; "Play again" starts a new bot game; a backend restart during the bot's turn; a + second browser watches a bot game from the live previews. diff --git a/docs/specs/game-bot/requirements.md b/docs/specs/game-bot/requirements.md new file mode 100644 index 0000000..f3dfb95 --- /dev/null +++ b/docs/specs/game-bot/requirements.md @@ -0,0 +1,74 @@ +# Game bot: requirements + +Status: spec, not implemented yet. Branch `feature/game-bot` (based on `develop`). +Scope: backend, `packages/core`, web app and mobile app. + +## Goals + +Players can always get a game: when no other player is looking for one, they can play against a bot. They choose +how strong the bot is, but within a level the bot is not always equally strong, so games against it differ from +each other. A game against the bot is a normal game: it has the clock, can be watched and is saved like any other. + +## Terms + +- **Bot**: a player run by the server. +- **Level**: what the player chooses: Easy, Medium, Hard or Master (Turkish: Kolay, Orta, Zor, Usta). +- **Strength**: a number from 0 to 100 that sets how well the bot plays. Each level is a range of strengths; the bot + takes a strength from it at the start of each game. +- **Bot game**: a game between a player and the bot. + +## Requirements + +### R1. Starting a bot game +- R1.1 When a player has been looking for an opponent for 10 seconds without one being found, the lobby offers a + game against the bot with the four levels to choose from. The search goes on while the offer is shown; if an + opponent is found first, the game with them starts as usual. +- R1.2 Choosing a level ends the search and starts a bot game at that level at once. +- R1.3 The home page also has a "Play with bot" button that opens the same choice of levels, without searching for + an opponent first. +- R1.4 The level chosen last is preselected the next time (kept on the device). +- R1.5 A player who has a game in progress cannot start a bot game; they are offered to continue that game, as when + they look for an opponent. +- R1.6 Who moves first is chosen at random, as in other games. + +### R2. Levels and strength +- R2.1 There are four levels, from weakest to strongest: Easy, Medium, Hard, Master. +- R2.2 At the start of each game the bot takes a random strength within the level's range. Within a game, it plays + each move with a strength that varies a little around that value. +- R2.3 Easy makes obvious mistakes and can be beaten by a beginner. Master plays close to perfectly and is rarely + beaten. +- R2.4 Each level is clearly stronger than the one below: played against each other by the server, the stronger + of two neighbouring levels wins at least 65% of the games, and Master wins at least 95% against Easy. +- R2.5 Two games at the same level from the same starting position do not play out the same way: in 20 such games + against the same sequence of the opponent's moves, the bot's first five moves are not all the same. + +### R3. How the bot plays +- R3.1 The bot plays by the same rules as players; its moves go through the same game logic. +- R3.2 The bot does not move instantly: it waits until the previous move's animation has finished, then about 1 to + 3 seconds more, a little longer at the higher levels. An extra turn is played the same way. +- R3.3 The bot has a clock like any player and its waiting counts on it. It never runs out of time. +- R3.4 The bot never leaves a game and is always shown as online. +- R3.5 Working out the bot's moves never delays other games: other players' moves are handled as fast as without + bots, even while many bot games are being played. + +### R4. During and after the game +- R4.1 The bot is shown as "Bot" with its level, e.g. "Bot · Medium" / "Bot · Orta", wherever players are shown: + the game page, live game previews, recent and popular games, and replays. +- R4.2 Everything that ends a game works as in other games: a player who leaves loses, a player whose time runs out + loses, a completed game is won by the player with more stones. +- R4.3 Bot games are saved like other games and appear in the player's recent games and can be replayed. +- R4.4 At the end of a bot game, the game over card offers "Play again", which starts a new bot game at the same + level. +- R4.5 Bot games can be watched by others and appear in live game previews like other games. +- R4.6 A bot game survives a backend restart: the bot goes on at the same strength, and if it was its turn, it moves. + +### R5. Mobile app +- R5.1 The mobile app has the offer in the lobby (R1.1, R1.2), the button on the home screen (R1.3) and shows the + bot's name with its level (R4.1). + +## Out of scope (TODO) +- Playing against the bot without a connection (a bot in the TypeScript engine on the device). +- Ratings or statistics; when they come, bot games should be kept apart from games against people. +- Hints or move analysis for players. +- Choosing an exact strength or who moves first. +- Rebuilding the game engine (`MancalaGame`, `Board`, the rules) on the compact board this feature adds. diff --git a/docs/specs/game-bot/tasks.md b/docs/specs/game-bot/tasks.md new file mode 100644 index 0000000..6e44062 --- /dev/null +++ b/docs/specs/game-bot/tasks.md @@ -0,0 +1,21 @@ +# Game bot: tasks + +Work through the tasks in order; each ends with its tests passing and a commit. Tick a task when it is committed. + +- [x] 1. **Spec**: requirements, design and tasks. +- [ ] 2. **Compact board**: `Position` in the engine package; tests against `MancalaGame` with the recorded parity + games and random games. +- [ ] 3. **Search and strength**: `BotLevel`, `BotSettings`, `BotStrength`, `BotSearch`, `BotPolicy` with unit + tests; `BotTournamentTest` (tag `calibration`) and its quick version; tune the evaluation weights and the + strength table until R2.4 holds, and write the final values into `design.md`. +- [ ] 4. **Bot games in the backend**: `new_bot_game`, `BackendGame.bot`, `playMove` split out of `onGameMove`, + `onTurnStarted`, bot ids in `GameStore`, `PublicIds` and the connection info; `BotPlayer` and its executor in + `Backend`, `BOT_SEARCH_THREADS`; `bot` in snapshots and their restore; unit tests. +- [ ] 5. **Saving**: `bot` in `GameRecord`, migration `003_bot`, the repository; integration tests. +- [ ] 6. **Core**: `channel_new_bot_game`, `BotLevel`, `BOT_LEVELS`, `BOT_OFFER_DELAY_MS`, `isBotId`, + `getBotLevel`, `getPlayerDisplayName` for bots, texts in English and Turkish; unit tests. +- [ ] 7. **Web app**: `BotLevelPicker`, the home page button, the lobby offer and `?bot=`, "Play again" after a bot + game, bot names everywhere players are shown. +- [ ] 8. **Mobile app**: lobby offer and `botLevel`, home screen button, bot names; type check and bundle. +- [ ] 9. **End to end and docs**: the Chrome tests from `design.md`; `README.md`, `apps/backend/README.md`, + `AGENTS.md` (the specs list, bot ids, the calibration test) and `TODO.md` (the out of scope items).