diff --git a/docs/architecture-and-flow-mermaid.md b/docs/architecture-and-flow-mermaid.md new file mode 100644 index 0000000..901f5b7 --- /dev/null +++ b/docs/architecture-and-flow-mermaid.md @@ -0,0 +1,328 @@ +# Architecture And Flow Diagrams + +This document captures the current project architecture and major runtime flows +in Mermaid format. The diagrams reflect the implementation in the repository, +not older planning notes. + +## Layered Architecture + +```mermaid +flowchart TB + Main["main/app_main"] + App["app_composition\nstartup sequencing and wiring"] + CP["control_plane\npolicy, lifecycle, workers, retry/backoff"] + Domain["domain\nbusiness contracts, adapters, semantic mapping"] + Platform["platform\nhardware, transport, storage, display, audio"] + HW["ESP32-S3 hardware\nLCD, audio codec, GPIO, Wi-Fi, flash"] + + Main --> App + App --> CP + CP --> Domain + Domain --> Platform + Platform --> HW +``` + +## Component Responsibility Map + +```mermaid +flowchart LR + subgraph AppComposition["app_composition"] + Boot["Plan A boot sequence"] + Wire["Runtime wiring"] + end + + subgraph ControlPlane["control_plane"] + Connectivity["Connectivity bootstrap and Wi-Fi policy"] + Workers["Runtime worker ownership"] + Events["System event dispatch"] + MultimodalPolicy["Multimodal session policy"] + Fortune["Daily fortune policy"] + RemoteLog["Remote log service"] + FactoryCli["Factory CLI diagnostics"] + end + + subgraph Domain["domain"] + ExpressionMap["LCD expression semantic mapping"] + RuntimeAdapters["Runtime facade adapters"] + Direction["Direction and calibration model"] + FortuneDomain["Birth profile and fortune contracts"] + UserConfig["LAN user config routes"] + end + + subgraph Platform["platform"] + LCD["ST7789 direct LCD backend"] + Audio["ES8311 audio pipeline"] + Network["Wi-Fi, BLE provisioning, HTTP, WebSocket"] + Storage["NVS, LittleFS, LogFS"] + GPIO["GPIO, stepper, console UART"] + end + + Boot --> Connectivity + Boot --> Workers + Wire --> Workers + Connectivity --> RuntimeAdapters + Workers --> Events + Workers --> RemoteLog + MultimodalPolicy --> RuntimeAdapters + Fortune --> FortuneDomain + Events --> ExpressionMap + FactoryCli --> RemoteLog + RuntimeAdapters --> LCD + RuntimeAdapters --> Audio + RuntimeAdapters --> Network + RuntimeAdapters --> Storage + RuntimeAdapters --> GPIO +``` + +## Current Boot Sequence + +```mermaid +sequenceDiagram + participant Main as app_main + participant App as app_composition + participant CP as control_plane + participant Domain as domain + participant Platform as platform + + Main->>App: app_composition_start_default_runtime() + App->>CP: control_plane_prepare_runtime_storage() + CP->>Domain: prepare storage through runtime adapters + Domain->>Platform: NVS / storage initialization + + App->>CP: control_plane_play_boot_sound(timeout) + CP->>Domain: domain_runtime_multimodal_audio_init() + Domain->>Platform: ES8311 / I2S / I2C audio init + CP->>Domain: domain_runtime_behavior_play_prompt_file() + Domain->>Platform: play prompt file + + App->>CP: control_plane_init_lcd_device() + CP->>Domain: domain_runtime_init_lcd_device() + Domain->>Platform: LCD GPIO/SPI/ST7789 init + + App->>CP: control_plane_start_lcd_render_worker() + CP->>CP: create cp_lcd_render task + CP->>Domain: domain_runtime_start_lcd_render_loop() + Domain->>Platform: lcd_direct_start_render_loop() + + App->>CP: control_plane_init() + CP->>Domain: platform/domain runtime initialization + Domain->>Platform: stepper, storage, behavior adapters + + App->>CP: control_plane_request_boot_expression() + CP->>Domain: request DOMAIN_EXPRESSION_IDLE + Domain->>Platform: queue LCD idle asset sequence + + opt boot turntable calibration enabled + App->>CP: control_plane_calibrate() + CP->>Domain: direction calibration policy + Domain->>Platform: stepper + calibration switch + end + + App->>CP: control_plane_bootstrap_connectivity() + CP->>Domain: Wi-Fi init / STA connect / BLE provisioning fallback + Domain->>Platform: Wi-Fi and provisioning primitives + + App->>CP: control_plane_start_runtime_workers() + CP->>CP: start model, websocket wait, LCD, Wi-Fi state, events, keys, remote log, fortune +``` + +## Runtime Worker Topology + +```mermaid +flowchart TB + Start["control_plane_start_runtime_workers()"] + + Start --> ModelQueue["ensure model command queue"] + Start --> ModelTask["cp_model_cmd"] + Start --> WsWait["WebSocket wait task"] + Start --> LcdTask["cp_lcd_render"] + Start --> WifiTask["cp_wifi_state"] + Start --> EventTask["cp_system_event"] + Start --> KeyTask["cp_key_input"] + Start --> RemoteLogTask["cp_remote_log"] + Start --> FortuneTask["cp_fortune\nif FORTUNE_ENABLE"] + + KeyTask --> EventTask + EventTask --> LcdTask + WsWait --> Multimodal["multimodal client/runtime"] + Multimodal --> ModelTask + Multimodal --> Audio["audio capture/playback"] + WifiTask --> LcdTask + RemoteLogTask --> LogServer["remote log server"] + FortuneTask --> FortuneApi["daily fortune API"] +``` + +## Connectivity Bootstrap Flow + +```mermaid +flowchart TD + Start["control_plane_bootstrap_connectivity"] + InitWiFi["domain_runtime_network_wifi_init(seed credentials)"] + HasCreds{"saved Wi-Fi credentials?"} + StaConnect["connect STA with feedback"] + StaOk{"STA connected?"} + KeepCreds{"error keeps credentials?"} + ClearCreds["clear saved Wi-Fi credentials"] + ResolvePop["resolve provisioning PoP\nserial number preferred"] + HasPop{"PoP available?"} + BleProv["start BLE provisioning"] + Done["return ESP_OK"] + Error["return error"] + + Start --> InitWiFi --> HasCreds + HasCreds -- yes --> StaConnect --> StaOk + StaOk -- yes --> Done + StaOk -- no --> KeepCreds + KeepCreds -- yes --> Done + KeepCreds -- no --> ClearCreds --> ResolvePop + HasCreds -- no --> ResolvePop + ResolvePop --> HasPop + HasPop -- yes --> BleProv --> Done + HasPop -- no --> Error +``` + +## LCD Expression Flow + +```mermaid +flowchart LR + CP["control_plane policy/event"] + DomainReq["domain_runtime_request_expression()"] + Resolve["domain_resolve_expression_platform_request()"] + Catalog["domain expression catalog\nIDLE, WIFI_DISCONNECTED,\nSERVER_ERROR, MUYU, CHOUQIAN, WUXING"] + PlatformReq["platform_lcd_request_presentation()"] + Pending["lcd_direct pending request"] + Render["cp_lcd_render loop"] + Draw["draw RLE asset frame\nfrom /littlefs/lcd"] + Panel["ST7789 panel"] + + CP --> DomainReq --> Resolve --> Catalog --> PlatformReq --> Pending --> Render --> Draw --> Panel + + Draw --> Delay["vTaskDelay(frame_ms)"] + Delay --> Render +``` + +## Key And Local Event Flow + +```mermaid +flowchart TD + GPIO["GPIO edge interrupt\nPTT GPIO15 / SW2 GPIO45"] + IsrQueue["key IRQ queue"] + KeyTask["cp_key_input\ndebounce and click state machine"] + SystemQueue["system event queue"] + EventTask["cp_system_event"] + PttState["PTT active state"] + Chouqian["SW2 short press\nDOMAIN_EXPRESSION_CHOUQIAN"] + Wuxing["SW2 double click\nDOMAIN_EXPRESSION_WUXING"] + Muyu["PTT double click\nDOMAIN_EXPRESSION_MUYU"] + + GPIO --> IsrQueue --> KeyTask --> SystemQueue --> EventTask + EventTask --> PttState + EventTask --> Chouqian + EventTask --> Wuxing + EventTask --> Muyu +``` + +## Multimodal Runtime Flow + +```mermaid +sequenceDiagram + participant CP as control_plane + participant Domain as domain + participant Platform as platform + participant WS as WebSocket server + participant Audio as ES8311 audio path + participant LCD as LCD expression + + CP->>Domain: prepare multimodal start + CP->>Domain: register event callback + CP->>Domain: register downstream audio callback + CP->>CP: start multimodal watchdog task + CP->>Domain: control_plane_multimodal_start() + Domain->>Platform: open WebSocket transport + Platform->>WS: connect and send Start directive + + WS-->>Platform: connected / task events + Platform-->>Domain: multimodal events + Domain-->>CP: event callback + + WS-->>Platform: downstream audio binary + Platform-->>Domain: downstream audio callback + Domain-->>CP: control_plane_downstream_audio_bridge() + CP->>Audio: play downstream audio + + alt transport/task/output error while STA connected + CP->>LCD: request DOMAIN_EXPRESSION_SERVER_ERROR + end + + alt server responding starts + CP->>Audio: stop uplink if running + end +``` + +## Remote Logging Flow + +```mermaid +flowchart TD + EspLog["ESP_LOGx / printf log"] + Hook["esp_log_set_vprintf hook"] + Filter["filter by CONFIG_TQ_REMOTE_LOG_MIN_LEVEL"] + RamQueue["RAM queue\nCONFIG_TQ_REMOTE_LOG_QUEUE_LEN"] + FlushTimer["periodic flush\nCONFIG_TQ_REMOTE_LOG_FLUSH_INTERVAL_MS"] + LogFS["/logfs/remote_logs\nLittleFS persistent queue"] + StaReady{"STA connected\nand realtime path idle?"} + Upload["HTTP POST JSON batch\nCONFIG_TQ_REMOTE_LOG_ENDPOINT"] + Server["remote_log_receiver.py\n/opt/logTools/logs"] + Backoff["exponential retry backoff"] + DropOld["trim old files\nCONFIG_TQ_REMOTE_LOG_MAX_FILES"] + CoreDump["detect flash coredump\nupload metadata report"] + + EspLog --> Hook --> Filter --> RamQueue + RamQueue --> FlushTimer --> LogFS --> DropOld + LogFS --> StaReady + CoreDump --> StaReady + StaReady -- yes --> Upload --> Server + StaReady -- no --> Backoff --> StaReady + Upload -- fail --> Backoff + Upload -- success --> Delete["delete uploaded local file"] +``` + +## Daily Fortune Flow + +```mermaid +flowchart TD + Task["cp_fortune task"] + WaitTime["wait for local time / next midnight / refresh request"] + WaitNetwork["wait for STA and multimodal readiness"] + Date["resolve local date"] + Fetch["control_plane_refresh_daily_fortune(date)"] + DomainFortune["domain fortune APIs\nbirth profile + fortune result"] + Http["HTTPS fortune API"] + Persist["persist fortune result"] + Notify["notify observers / UI configuration server"] + Retry["retry delay on failure"] + + Task --> WaitTime --> WaitNetwork --> Date --> Fetch + Fetch --> DomainFortune --> Http + Http -- success --> Persist --> Notify --> WaitTime + Http -- failure --> Retry --> WaitTime +``` + +## Factory CLI Diagnostics Flow + +```mermaid +flowchart LR + UART["UART0 serial input"] + CLI["factory CLI parser"] + Identity["identity / auth commands"] + LogStatus["log status"] + LogFlush["log flush"] + LogClear["log clear"] + RemoteLog["control_plane_remote_log_*"] + UARTOut["UART0 JSON/OK/ERROR output"] + + UART --> CLI + CLI --> Identity --> UARTOut + CLI --> LogStatus --> RemoteLog --> UARTOut + CLI --> LogFlush --> RemoteLog --> UARTOut + CLI --> LogClear --> RemoteLog --> UARTOut +``` diff --git a/docs/firmware-pin-usage.md b/docs/firmware-pin-usage.md new file mode 100644 index 0000000..e6c7e07 --- /dev/null +++ b/docs/firmware-pin-usage.md @@ -0,0 +1,117 @@ +# Firmware Pin Usage + +This document summarizes the GPIOs used by the current firmware configuration. +The primary source of truth is `sdkconfig.defaults`; generated `sdkconfig` +differences are called out where they affect runtime behavior. Signal direction +is from the ESP32-S3 MCU perspective. + +## Configured Firmware Pins + +| GPIO | Signal / Config | Direction | Subsystem | Runtime use | Electrical / logic notes | +| ---: | --- | --- | --- | --- | --- | +| 1 | `CONFIG_MULTIMODAL_AUDIO_I2C_SDA_GPIO` | Bidirectional | Audio codec | ES8311 I2C SDA | I2C port 0, codec control bus | +| 2 | `CONFIG_MULTIMODAL_AUDIO_I2C_SCL_GPIO` | Output | Audio codec | ES8311 I2C SCL | I2C port 0, codec control bus | +| 3 | `CONFIG_TQ_SCREEN_RESET_PIN` | Output | LCD | ST7789 panel reset | Set high after GPIO init; `-1` disables hardware reset | +| 6 | `CONFIG_TQ_SCREEN_SPI_CS_GPIO` | Output | LCD | ST7789 SPI chip select | SPI2 host | +| 7 | `CONFIG_TQ_SCREEN_DC_GPIO` | Output | LCD | ST7789 data/command select | Initialized low | +| 9 | `CONFIG_TQ_SCREEN_BACKLIGHT_GPIO` | Output | LCD | LCD backlight enable | Active-high by default | +| 12 | `CONFIG_MULTIMODAL_AUDIO_PA_GPIO` | Output | Audio amplifier | PA enable / NS4150B control | Managed by codec GPIO interface | +| 13 | `CONFIG_TQ_STEPPER_IN1_GPIO` | Output | Turntable | ULN2003 stepper IN1 | Batched GPIO phase updates | +| 14 | `CONFIG_TQ_STEPPER_IN2_GPIO` | Output | Turntable | ULN2003 stepper IN2 | Batched GPIO phase updates | +| 15 | `CONFIG_MULTIMODAL_PTT_BUTTON_GPIO` | Input | Keys / multimodal | Push-to-talk button K2 | Active-low, internal pull-up enabled | +| 17 | `CONFIG_TQ_SCREEN_SPI_MOSI_GPIO` | Output | LCD | ST7789 SPI MOSI / SDA | SPI2 host | +| 18 | `CONFIG_TQ_SCREEN_SPI_CLK_GPIO` | Output | LCD | ST7789 SPI clock / SCK | SPI2 host, 20 MHz default | +| 21 | `CONFIG_TQ_STEPPER_IN3_GPIO` | Output | Turntable | ULN2003 stepper IN3 | Batched GPIO phase updates | +| 38 | `CONFIG_MULTIMODAL_AUDIO_I2S_MCLK_GPIO` | Output | Audio codec | ES8311 I2S MCLK | Enabled by `CONFIG_MULTIMODAL_AUDIO_USE_MCLK=y` | +| 39 | `CONFIG_MULTIMODAL_AUDIO_I2S_BCLK_GPIO` | Output | Audio codec | ES8311 I2S BCLK / SCLK | I2S port 0 | +| 40 | `CONFIG_MULTIMODAL_AUDIO_I2S_DOUT_GPIO` | Output | Audio codec | MCU PCM output to ES8311 DSDIN | I2S port 0 | +| 41 | `CONFIG_MULTIMODAL_AUDIO_I2S_WS_GPIO` | Output | Audio codec | ES8311 I2S WS / LRCK | I2S port 0 | +| 42 | `CONFIG_MULTIMODAL_AUDIO_I2S_DIN_GPIO` | Input | Audio codec | MCU PCM input from ES8311 ASDOUT | I2S port 0 | +| 45 | `CONFIG_TQ_SLOT_SWITCH_GPIO` | Input | Slot / key input | SW2 slot-machine feature switch | Active-low, internal pull-up enabled | +| 47 | `CONFIG_TQ_STEPPER_IN4_GPIO` | Output | Turntable | ULN2003 stepper IN4 | Batched GPIO phase updates | +| 48 | `CONFIG_TQ_CALIBRATION_SWITCH_GPIO` | Input | Turntable calibration | SW1 direction calibration microswitch | Active-low, internal pull-up enabled | + +## Disabled Or Not Connected In Defaults + +| Config | Value | Meaning | +| --- | ---: | --- | +| `CONFIG_TQ_SCREEN_SPI_MISO_GPIO` | `-1` | LCD MISO is not connected or not used. | +| `CONFIG_TQ_POWER_HOLD_GPIO` | `-1` | Power-hold output is disabled. | +| `CONFIG_TQ_SLOT_SWITCH_PULLDOWN` | unset | SW2 uses pull-up only. | +| `CONFIG_TQ_CALIBRATION_SWITCH_PULLDOWN` | unset | Calibration switch uses pull-up only. | +| `CONFIG_MULTIMODAL_PTT_BUTTON_PULLDOWN` | unset | PTT button uses pull-up only. | + +## Console / Factory CLI + +The factory CLI uses `UART_NUM_0` through the ESP-IDF UART driver without +project-specific GPIO remapping. The board reference map routes the debug UART +to the `RXD0` / `TXD0` nets on connector `J2`. + +| Interface | Direction | Use | +| --- | --- | --- | +| UART0 RX | Input | Factory CLI input and serial monitor input | +| UART0 TX | Output | ESP log output, factory CLI replies, serial monitor output | + +## External Memory Pins + +The project enables external PSRAM in octal mode. These are generated ESP-IDF +memory-interface settings in `sdkconfig`, not normal application GPIOs. + +| GPIO | Config | Direction | Use | Notes | +| ---: | --- | --- | --- | --- | +| 26 | `CONFIG_SPIRAM_CS_IO` / `CONFIG_DEFAULT_PSRAM_CS_IO` | Memory bus | PSRAM CS | Reserved by ESP32-S3 memory interface | +| 30 | `CONFIG_SPIRAM_CLK_IO` / `CONFIG_DEFAULT_PSRAM_CLK_IO` | Memory bus | PSRAM clock | Reserved by ESP32-S3 memory interface | + +Flash pins are controlled by the ESP32-S3 flash interface and are not assigned +by project GPIO configuration. + +## Board Nets Present But Not Used By Current Firmware + +These nets exist in `docs/gpio-map.md`, but the current firmware does not assign +them to an active project feature. + +| GPIO / Net | Board function | Firmware status | +| --- | --- | --- | +| GPIO0 / K1 | Switch to GND, commonly boot/download related | Not consumed by application code | +| GPIO4 | Unassigned | Not used | +| GPIO5 | Unassigned | Not used | +| GPIO8 | Unassigned | Not used | +| GPIO10 / `SWING_PWM` | Swing coil control interface | No current firmware driver/config found | +| GPIO11 | Unassigned | Not used | +| GPIO16 | Unassigned | Not used | +| GPIO35 | Unassigned | Not used | +| GPIO36 | Unassigned | Not used | +| GPIO37 | Unassigned | Not used | +| GPIO46 | Unassigned | Not used | +| GPIO19 / USB D- | USB | Board/system function; not configured by application code | +| GPIO20 / USB D+ | USB | Board/system function; not configured by application code | + +## Current Generated `sdkconfig` Difference + +`sdkconfig.defaults` enables the key input task: + +```text +CONFIG_TQ_KEY_INPUT_ENABLE=y +``` + +The current generated `sdkconfig` has it disabled: + +```text +# CONFIG_TQ_KEY_INPUT_ENABLE is not set +``` + +This means the defaults describe the intended project pin usage, while the +current generated build configuration may not actively start the interrupt key +input task unless `sdkconfig` is regenerated or updated. + +## Source Files + +- `sdkconfig.defaults` +- `sdkconfig` +- `main/Kconfig.projbuild` +- `docs/gpio-map.md` +- `components/platform/src/lcd_direct.c` +- `components/platform/src/platform_stepper_driver.c` +- `components/platform/src/multimodal/audio_pipeline/audio_pipeline_codec_impl.c` +- `components/control_plane/src/runtime/control_plane_key_input.c` +- `components/platform/src/platform_console_driver.c` diff --git a/docs/国芯微离线语音协议_v1.3.7.pdf b/docs/国芯微离线语音协议_v1.3.7.pdf new file mode 100644 index 0000000..ac062bb Binary files /dev/null and b/docs/国芯微离线语音协议_v1.3.7.pdf differ