Halit Aksoy
f7a76c861e
The bot in the backend README with the calibration command, AGENTS.md notes on bot ids and test tags, and the open work in TODO.md.
240 lines
18 KiB
Markdown
240 lines
18 KiB
Markdown
# Game bot: design
|
||
|
||
See `requirements.md` for what is built; requirement ids (R1.1, …) are referenced below. Builds on `game-clock`
|
||
(the clock), `game-history` (saved games, live previews) and `quarkus-backend` (the Java backend and its engine).
|
||
|
||
## Decisions
|
||
|
||
| Decision | Reason |
|
||
|---|---|
|
||
| The bot runs in the backend, in Java | The rules are already in Java (`app.playmancala.engine`) and kept equal to the TypeScript engine by `EngineParityTest`. A bot game is then an ordinary server game: the clock, saving, spectators, previews and restarts work without changes in the clients (R4). A bot on the device would only add offline play, at the cost of a second bot in TypeScript (out of scope). |
|
||
| A compact board, `Position` (`int[14]` and whose turn it is), in the engine package | Searching means playing millions of moves; `MancalaGame` builds `Pit` objects, lists and animation steps for each, which is far too slow. `Position` plays a move into an existing array without allocating. It is in `engine`, not `bot`, so other code can use it too, e.g. checking a move before running it (see `TODO.md`, Security). It is tested against `MancalaGame` with the recorded parity games and random games. |
|
||
| `MancalaGame` is not rebuilt on `Position` now | `MancalaGame` also produces the game steps clients animate, the history and the JSON the Node backend wrote, and it must stay equal to the TypeScript engine. Rebuilding it is a separate refactor once `Position` has proven itself (out of scope). |
|
||
| Negamax with alpha-beta pruning, limited by depth and a node budget, not by time | Mangala is small (6 moves at most per turn), so alpha-beta is strong enough: at depth 12 the bot plays close to perfectly. Limits that do not depend on time make a move depend only on the position, the strength and the seed, so a game can be reproduced for debugging; a slower machine only takes longer. |
|
||
| Strength (0–100) sets three things: search depth, how much the bot prefers the best move (softmax temperature) and how often it plays a random move | Depth alone gives few, coarse steps and a predictable bot. With softmax, a weaker bot more often plays a move that is nearly as good, which looks like a human mistake; the random moves make Easy beatable by beginners (R2.3). Interpolating a table gives smooth steps between levels. |
|
||
| Each level is a range of strengths; the strength is drawn per game and varies by ±5 per move | R2.2. The ranges have gaps between them so neighbouring levels stay clearly apart (R2.4). |
|
||
| Randomness comes from a per-game seed and the move number | The bot's choice of a move is reproducible from the saved game, and a restart does not change how it plays (R4.6). The seed and strength are kept with the live game and the saved game. |
|
||
| Searching runs on its own small thread pool; the result goes back to the game loop, which checks that the game is still waiting for that move | The game loop must never wait (R3.5, `AGENTS.md`). The pool has a fixed number of threads, so many bot games share the CPU instead of starving the loop and the WebSocket threads. A result is dropped if the game ended or moved on meanwhile. |
|
||
| The bot's waiting time is computed on the server: the previous move's animation (300 ms per step, as `PitAnimator`) plus a random 1–3 seconds | R3.2. Clients do not need to know about bots for this. Searching happens during the wait, so the time it takes is not added to it. |
|
||
| Bot ids are `bot:<level>`, e.g. `bot:medium` | One stable id per level reads well in saved games and lets later statistics group games by level. It is public and cannot belong to a person: Keycloak ids start with `kc:` and legacy ids must be UUIDs (`RtmtServer`). Because one id plays many games at once, `GameStore` does not map it to a game. Clients recognise bots by the id, so `PlayerSummary` keeps its shape. |
|
||
| The offer to play the bot (R1.1) is a timer in the client | The server does not need to know how long someone has waited. Matchmaking stays as it is; choosing a level sends a new message that also ends the search on the server. |
|
||
| Bot games are ordinary games everywhere: clock, saving, spectators, live previews, popular games | R4. Fewer special cases. The bot's moves go through the same code as a player's moves (R3.1). |
|
||
|
||
## Shared types (`packages/core`)
|
||
|
||
```ts
|
||
// rtmt/channel_names.ts
|
||
export const channel_new_bot_game = "new_bot_game"; // client -> server: { level: BotLevel }
|
||
|
||
// models/Bot.ts
|
||
export type BotLevel = "easy" | "medium" | "hard" | "master";
|
||
export const BOT_LEVELS: BotLevel[] = ["easy", "medium", "hard", "master"];
|
||
export const BOT_OFFER_DELAY_MS = 10_000; // R1.1
|
||
export function isBotId(userId: string | undefined): boolean; // starts with "bot:"
|
||
export function getBotLevel(userId: string | undefined): BotLevel | undefined;
|
||
export function isBotLevel(value: unknown): value is BotLevel;
|
||
export function getBotLevelTextKey(level: BotLevel): string; // e.g. BotLevelMedium
|
||
export function getLastBotLevel(storage): BotLevel; // Medium until one is chosen (R1.4)
|
||
export function setLastBotLevel(storage, level: BotLevel): void;
|
||
```
|
||
|
||
- `getPlayerDisplayName(player, t)` takes `userId` too (every caller has it) and returns e.g. "Bot · Orta" for
|
||
bots (R4.1).
|
||
- Texts (en, tr): `PlayWithBot` "Play with bot" / "Bota karşı oyna", `NoOpponentYet` "No opponent yet. Play with
|
||
the bot?" / "Henüz rakip bulunamadı. Bota karşı oynamak ister misin?", `ChooseBotLevel` "Choose how strong the bot
|
||
is" / "Botun ne kadar güçlü olacağını seç", `Bot` "Bot" / "Bot", `BotLevelEasy`,
|
||
`BotLevelMedium`, `BotLevelHard`, `BotLevelMaster` "Easy", "Medium", "Hard", "Master" / "Kolay", "Orta", "Zor",
|
||
"Usta", `PlayAgain` "Play again" / "Tekrar oyna".
|
||
|
||
## Backend
|
||
|
||
### Modules
|
||
```
|
||
engine/Position.java compact board and the rules on it
|
||
bot/BotLevel.java the levels, their wire names and strength ranges
|
||
bot/BotSettings.java record (level, strength, seed); kept with a bot game
|
||
bot/BotStrength.java strength -> depth, temperature, random move chance
|
||
bot/BotSearch.java negamax with alpha-beta; scores of the root moves
|
||
bot/BotPolicy.java chooses a move: strength, search, softmax, random moves
|
||
bot/BotPlayer.java plays the bot's turns in live games
|
||
db/migrations/003_bot.sql bot column
|
||
```
|
||
`engine` and `bot` (except `BotPlayer`) have no Quarkus or game loop code, so they can be tested and run on their
|
||
own.
|
||
|
||
### `Position`
|
||
```java
|
||
public final class Position {
|
||
// CommonBoard layout: player 1's pits 0-5, bank 6; player 2's pits 7-12, bank 13.
|
||
public static Position initial(boolean player1ToMove);
|
||
public static Position of(int[] stones, boolean player1ToMove);
|
||
public static Position of(MancalaGame game);
|
||
public int[] stones(); // a copy
|
||
public boolean isPlayer1ToMove();
|
||
public int legalPits(int[] into); // pit numbers 0-5 of the mover's non-empty pits; returns the count
|
||
// Plays pit 0-5 of the mover into `into` (may be this). Returns true if the mover plays again.
|
||
public boolean play(int pit, Position into);
|
||
public boolean isOver(); // one side is empty
|
||
public int bankDifference(); // mover's bank minus the opponent's
|
||
}
|
||
```
|
||
`play` applies the rules in the engine's order: sowing (one stone goes to the next pit, otherwise the first stone
|
||
goes back into the played pit; the opponent's bank is skipped), last stone in an own empty pit, even count in an
|
||
opponent's pit, last stone in the own bank (extra turn) or the turn passes, and clearing the board when a side is
|
||
empty.
|
||
|
||
### Search (`BotSearch`)
|
||
- Negamax with alpha-beta. A move that gives an extra turn keeps the side to move, so its child's score is not
|
||
negated.
|
||
- Scores are from the mover's view, in hundredths of a stone. An ended game scores
|
||
`±100_000 + 100 × final bank difference`; a game where a bank holds more than 24 stones is decided and scores as
|
||
ended. Other leaves: `100 × bank difference`. A term for the stones on each side (`PIT_STONE_WEIGHT`) is there
|
||
but set to 0: the levels met R2.4 without it, and stones on a side may still go to either player.
|
||
- Move order: extra turns first, then by the stones a move brings to the bank. Each node plays its moves once to
|
||
order them, into arrays kept per ply, so searching allocates nothing.
|
||
- Iterative deepening up to the depth; each root move is searched with a full window so every root move gets an
|
||
exact score (the softmax needs them). If the node budget (3 million) runs out, the scores of the last completed
|
||
depth are used; depth 1 always completes.
|
||
|
||
### Strength (`BotLevel`, `BotStrength`)
|
||
|
||
| Level | Wire name | Strength |
|
||
|---|---|---|
|
||
| Easy | `easy` | 5–25 |
|
||
| Medium | `medium` | 30–55 |
|
||
| Hard | `hard` | 60–80 |
|
||
| Master | `master` | 85–100 |
|
||
|
||
Interpolated linearly between rows (depth rounded):
|
||
|
||
| Strength | Depth | Temperature (stones) | Random move chance |
|
||
|---|---|---|---|
|
||
| 0 | 1 | 4 | 35% |
|
||
| 25 | 2 | 2 | 15% |
|
||
| 50 | 4 | 1 | 5% |
|
||
| 75 | 7 | 0.4 | 1% |
|
||
| 100 | 16 | 0.05 | 0% |
|
||
|
||
The first table had depth 12 at strength 100 and already met R2.4. Searching deeper still helped: with best moves
|
||
only, depth 16 scored 64% against depth 12 (depth 14: 54%), at about 60 ms per move on average and 1 s at most
|
||
without a budget. So the top row is 16, and the node budget keeps the slowest moves near 0.2 s.
|
||
|
||
`BotTournamentTest` with these values, 200 games per pair (October 2026, 24 cores, one thread):
|
||
|
||
| Pair | Won | Lost | Drawn | Score | Stronger level's time per move |
|
||
|---|---|---|---|---|---|
|
||
| Medium vs Easy | 155 | 34 | 11 | 0.80 | < 0.1 ms |
|
||
| Hard vs Medium | 158 | 29 | 13 | 0.82 | 0.2 ms |
|
||
| Master vs Hard | 177 | 16 | 7 | 0.90 | 20 ms |
|
||
| Master vs Easy | 200 | 0 | 0 | 1.00 | 22 ms |
|
||
|
||
### Choosing a move (`BotPolicy`)
|
||
```java
|
||
static int chooseMove(Position position, BotSettings bot, int moveNumber); // pit 0-5
|
||
```
|
||
1. A random source from `bot.seed()` and `moveNumber` (number of moves played in the game).
|
||
2. This move's strength: `bot.strength()` ±5, kept within 0–100.
|
||
3. With the random move chance: a random legal pit.
|
||
4. Otherwise search to the depth and pick a root move with probability ∝ `exp(score / (temperature × 100))`.
|
||
5. With a single legal pit, play it without searching.
|
||
|
||
### Bot games in the game logic (`GameManager`)
|
||
- New channel `new_bot_game` with `{ level }`. An unknown level is logged and ignored. A player with a game in
|
||
progress gets `on_new_game_rejected` (R1.5). Otherwise `matchMaker.cancel(userId)`, then a game with the player
|
||
as player 1 and `bot:<level>` as player 2, a random first mover (R1.6), and
|
||
`bot = new BotSettings(level, random strength in the level's range, random seed)`; then `startGame` as usual.
|
||
- `BackendGame` gains `BotSettings bot` (null for games between people) and `botId()`.
|
||
- The bot's `PlayerSummary` is `(bot:<level>, "Bot", false)`.
|
||
- `onGameMove` is split: the socket handler finds the game and calls `playMove(game, playerId, pitIndex)`, which
|
||
holds today's logic (clock check, engine, clock charge, updates, ending). The bot calls the same method (R3.1).
|
||
- `onTurnStarted(Consumer<BackendGame>)`: called after a game starts, after a move that does not end the game
|
||
(including an extra turn) and for each restored game.
|
||
- Connection info shows the bot online (R3.4); messages to the bot's id are not sent.
|
||
- `GameStore.setGameByUser` and `removeGameByPlayer` ignore bot ids.
|
||
- `PublicIds.publicUserId` returns bot ids as they are.
|
||
|
||
### `BotPlayer`
|
||
```java
|
||
public BotPlayer(GameManager gameManager, GameStore gameStore, Scheduler loop, Executor searchExecutor,
|
||
RandomGenerator random) // random: thinking times only; used on the loop
|
||
```
|
||
Listens to `onTurnStarted`. When it is the bot's turn and no move is being worked out for that game:
|
||
1. On the loop: copy the position (`Position.of`), note the move number and compute
|
||
`readyAt = now + animationMs(last move) + thinkMs(level)`. `animationMs` is 300 ms per `GAME_STEP_GAME_MOVE` step
|
||
of the last move plus 300 ms; before the first move it is 1.5 s, so the game page has opened. `thinkMs` is random: Easy 0.8–1.8 s, Medium 1.0–2.2 s, Hard 1.2–2.6 s, Master
|
||
1.4–3.0 s. With less than 30 s on the bot's clock it is 0.3 s (R3.3).
|
||
2. On `searchExecutor`: `BotPolicy.chooseMove`. If it throws, the error is logged and the first legal pit is used,
|
||
so the game never waits for a bot that failed.
|
||
3. Back on the loop (`loop.execute`, then `loop.schedule` for what is left until `readyAt`): if the game is still
|
||
live, it is the same `BackendGame`, the move number has not changed and it is the bot's turn, call
|
||
`gameManager.playMove(game, botId, pit)`; otherwise drop the result.
|
||
|
||
Games being worked on are kept in a set by identity, so a game restored with the same id is a new game for it.
|
||
|
||
`searchExecutor` is a fixed pool of `BOT_SEARCH_THREADS` platform threads (default: half the processors, at least
|
||
1), created and closed in `Backend`. Unit tests pass a direct executor and `ManualScheduler`.
|
||
|
||
### Restore (R4.6)
|
||
- `LiveGameSnapshot` gains `bot` (`@JsonInclude(NON_NULL)`); the snapshot stays version 2, since the field is
|
||
optional. `fromSnapshot` restores it. A snapshot with a bot player but no `bot` field (not expected) gets the
|
||
middle strength of the level and a new seed.
|
||
- `restoreLiveGames` calls the turn listeners, so a bot whose turn it was moves.
|
||
|
||
### Saving (R4.3)
|
||
- `GameRecord` gains `bot` (`@JsonInclude(NON_NULL)`): `{ level, strength, seed }`, sent with `GET /game/:id`;
|
||
clients do not use it. It allows tuning the levels from real games later.
|
||
- Migration `003_bot`: `ALTER TABLE games ADD COLUMN bot jsonb;` (NULL for games between people).
|
||
`PostgresGameHistoryRepository` writes and reads it.
|
||
|
||
### Configuration
|
||
`BOT_SEARCH_THREADS` (default: half the processors, at least 1).
|
||
|
||
## Web app
|
||
- `components/BotLevelPicker.tsx`: the four levels as buttons; the last chosen level is highlighted. It is kept with
|
||
`getLastBotLevel` / `setLastBotLevel` from core in the app's storage (R1.4).
|
||
- Home page: a "Play with bot" button next to "New Game" opens `BotLevelMenu`, a menu of the levels like the account
|
||
menu, with the last level checked; choosing a level goes to `/loby?bot=<level>` (R1.3).
|
||
- `LobyPage`: with `?bot=<level>`, it sends `new_bot_game` instead of `new_game`. Otherwise, after
|
||
`BOT_OFFER_DELAY_MS` without a game, it shows `NoOpponentYet` and the picker below the search; choosing a level
|
||
sends `new_bot_game` (R1.1, R1.2). The existing `on_game_start` and `on_new_game_rejected` handling stays.
|
||
- `GamePage` / `GameOverCard`: in a bot game, the main button is "Play again" and goes to `/loby?bot=<level>` with
|
||
the bot's level (R4.4).
|
||
- Bot names: every place that shows a player uses `getPlayerDisplayName` with the player's `userId` (R4.1).
|
||
|
||
## Mobile app
|
||
- `LobyScreen`: the same offer after `BOT_OFFER_DELAY_MS`, and a `botLevel` route parameter that sends
|
||
`new_bot_game` at once.
|
||
- `HomeScreen`: a "Play with bot" button with the level choice; the last level is kept in MMKV.
|
||
- Bot names through `getPlayerDisplayName`.
|
||
|
||
## Testing
|
||
- `Position`: replays the recorded parity games (`/engine/recorded-games.json`) and a few thousand random games
|
||
move by move against `MancalaGame`, comparing stones, whose turn it is and the end.
|
||
- `BotSearch`: finds an extra turn, a capture and a win in one or two moves; the same position, depth and budget
|
||
give the same scores; the node budget is respected.
|
||
- `BotStrength` and `BotLevel`: interpolation, the ranges, strengths drawn stay in range.
|
||
- `BotPolicy`: the same seed and move number give the same move; random moves only at the expected rate; a
|
||
single legal pit is played; R2.5 with a fixed opponent.
|
||
- Calibration: `BotTournamentTest`, tagged `calibration` and left out of `./mvnw test`
|
||
(`./mvnw test -Dgroups=calibration`), plays 200 games for each pair of neighbouring levels and Master against
|
||
Easy, alternating the first mover, and fails if R2.4 does not hold. It also reports the average search time per
|
||
level. A quick version (20 games, Master against Easy) runs with the normal tests.
|
||
- `BotPlayer` with `FakeRtmt`, `ManualScheduler` and a direct executor: the bot moves after the delay; extra turns;
|
||
a game that ended or moved on while searching drops the result; low time shortens the wait; a restored game where
|
||
it is the bot's turn gets a move. (The fallback for a failing search has no test: `BotPolicy` is static and cannot
|
||
be made to fail from outside.)
|
||
- `GameManager`: `new_bot_game` starts a game with the bot and ends the search; it is refused during a game;
|
||
unknown levels are ignored; the bot shows as online and gets no messages; the saved record keeps the bot id and
|
||
`bot`; leaving and running out of time end the game as usual.
|
||
- Postgres repository: the `bot` column is saved and read. Snapshots with and without `bot` restore.
|
||
- End to end in Chrome with `GAME_CLOCK_MS=60000`: the offer appears after 10 s in the lobby and starts a game;
|
||
"Play with bot" on the home page; the bot moves after the animation; the game ends and shows in recent games and
|
||
the replay as "Bot · <level>"; "Play again" starts a new bot game; a backend restart during the bot's turn; a
|
||
second browser watches a bot game from the live previews.
|
||
|
||
Checked end to end in Chrome on 2026-10-16 with the default clock: the home menu and the lobby offer (shown after
|
||
10.3 s) start games at the chosen level; "Bot · Easy" / "Bot · Hard" on the game page, in recent games and in a
|
||
second browser's live previews; an Easy game played to the end (the bot waited 1.45–4 s per move, longer after long
|
||
animations); "Play again" starts a new game at the same level; the backend stopped right after a Master game started
|
||
and the bot made its first move after the restart.
|