多模态

This commit is contained in:
admin
2026-02-24 21:36:14 +08:00
parent e1e0ac8929
commit 68840e6ea4
23 changed files with 3368 additions and 44 deletions

View File

@@ -1,8 +1,8 @@
# ESP32-S3 LYF Printer Controller (Wi-Fi REST API)
# ESP32-S3 TQ Printer Controller (Wi-Fi REST API)
ESP32-S3 works as a Wi-Fi REST controller and replaces Android App logic:
- REST client -> ESP32-S3 (`esp_http_server`)
- ESP32-S3 -> BLE printer (`lyfPrinter`, `FFF2` write / `FFF1` notify)
- ESP32-S3 -> BLE printer (`TQPrinter` / `lyfPrinter`, `FFF2` write / `FFF1` notify)
- Command compatibility:
- Print path: `0x00~0x07`
- OTA path: `0xA0~0xA4`
@@ -15,6 +15,10 @@ ESP32-S3 works as a Wi-Fi REST controller and replaces Android App logic:
- Async print queue with jobs (`queued/running/success/failed/canceled`)
- Printer precheck (paper / battery / temperature)
- Built-in UTF-8 text rendering with Chinese support (GB2312 character set, 16x16 bitmap)
- Tap2talk multimodal voice interaction over DashScope WebSocket:
- ES8311 microphone/speaker via `esp_codec_dev`
- raw-opus uplink/downlink via `esp_audio_codec`
- WebSocket transport via `esp_websocket_client`
- Android-compatible print flow:
- `0x00` power on
- `0x03` set print param
@@ -50,6 +54,12 @@ ESP32-S3 works as a Wi-Fi REST controller and replaces Android App logic:
- `POST /v1/ota/erase_page`
- `POST /v1/ota/write_frame`
- `POST /v1/ota/upgrade`
- Voice:
- `GET /v1/voice/status`
- `POST /v1/voice/session/start`
- `POST /v1/voice/session/stop`
- `POST /v1/voice/tap/start`
- `POST /v1/voice/tap/cancel`
## Build
@@ -66,13 +76,23 @@ idf.py build
## Config
In `menuconfig -> LYF Controller Config`:
- `LYF_WIFI_SSID`
- `LYF_WIFI_PASSWORD`
- `LYF_HTTP_PORT`
- `LYF_API_KEY` (optional)
In `menuconfig -> TQ Controller Config`:
- `TQ_WIFI_SSID`
- `TQ_WIFI_PASSWORD`
- `TQ_HTTP_PORT`
- `TQ_API_KEY` (optional)
- Voice WebSocket auth and app fields:
- `TQ_VOICE_API_KEY`
- `TQ_VOICE_WORKSPACE_ID`
- `TQ_VOICE_APP_ID`
- Voice audio and codec fields:
- `TQ_VOICE_SAMPLE_RATE`
- `TQ_VOICE_OPUS_FRAME_MS`
- `TQ_VOICE_OPUS_BITRATE_KBPS`
- `TQ_VOICE_I2C_*` / `TQ_VOICE_I2S_*`
- `TQ_VOICE_CODEC_*`
When `LYF_API_KEY` is not empty, each request must include:
When `TQ_API_KEY` is not empty, each request must include:
```text
X-API-Key: <your-key>
@@ -117,7 +137,9 @@ curl http://<esp-ip>/v1/health
```bash
curl -X POST http://<esp-ip>/v1/printer/connect \
-H 'Content-Type: application/json' \
-d '{"name":"lyfPrinter","timeout_ms":15000}'
-d '{"name":"TQPrinter","timeout_ms":15000}'
# Legacy device name is also supported:
# -d '{"name":"lyfPrinter","timeout_ms":15000}'
```
Auto-match a compatible printer (by BLE service `0xFFF0`):
@@ -170,7 +192,7 @@ curl -X POST http://<esp-ip>/v1/print/qr \
curl -X POST http://<esp-ip>/v1/print/text \
-H 'Content-Type: application/json' \
-d '{
"text":"欢迎使用LYF打印机\\n订单号: A1024\\n谢谢惠顾",
"text":"欢迎使用TQ打印机\\n订单号: A1024\\n谢谢惠顾",
"density":"中等",
"scale":2,
"line_spacing":2,
@@ -183,7 +205,7 @@ curl -X POST http://<esp-ip>/v1/print/text \
curl -X POST http://<esp-ip>/v1/print/receipt \
-H 'Content-Type: application/json' \
-d '{
"title":"LYF FOOD",
"title":"TQ FOOD",
"density":"中等",
"footer":"Thanks!",
"items":[
@@ -248,6 +270,37 @@ curl -X POST http://<esp-ip>/v1/ota/upgrade \
}'
```
### Voice tap2talk
Start session:
```bash
curl -X POST http://<esp-ip>/v1/voice/session/start
```
Start one tap2talk round:
```bash
curl -X POST http://<esp-ip>/v1/voice/tap/start
```
Cancel current tap2talk round:
```bash
curl -X POST http://<esp-ip>/v1/voice/tap/cancel
```
Get voice status:
```bash
curl http://<esp-ip>/v1/voice/status
```
Stop session:
```bash
curl -X POST http://<esp-ip>/v1/voice/session/stop
```
## Notes
- BLE side is central/client role, not printer peripheral role.
@@ -257,6 +310,7 @@ curl -X POST http://<esp-ip>/v1/ota/upgrade \
- `/v1/print/text` and `/v1/print/receipt` now support UTF-8 Chinese via embedded 16x16 GB2312 glyphs.
- Characters outside embedded glyph set are rendered as square fallback boxes.
- QR encoding uses embedded `qrcodegen` (Project Nayuki C implementation).
- Tap2talk uses server-side VAD and keeps uploading 100ms raw-opus frames in `Listening` state (silence when user is not tapping), matching `docs/websocket.md` requirements.
## Font Assets