docs(agents): document code runner and fix clippy/biome lint
文档: - AGENTS.md 新增「Code Runner」架构章节（三层架构 + WASM 可见性规则 + governor 0.8 无 per_day 的处理 + runner 镜像契约） - Environment 段补齐 CODE_RUNNER_* 与 RATE_LIMIT_CODE_EXEC_* 环境变量 Lint 修复（clippy --all-targets --all-features -D warnings 全绿）: - runner.rs/runner.rs: use_signal 冗余闭包 → 直接传 fn - runner_config.rs: &*RUNNER_CONFIG 去 deref;assert_eq!(...false/true) → assert! - markdown.rs: 局部变量 /// 改 //;Option.map().flatten() → and_then - languages.rs: 删除从未读取的 LanguageDef.name 字段（语言名即 map key） - docker.rs: start_container if let Err → ?;timeout match → is_err(); 嵌套 if 合并（修 task 2 遗留 + clippy 1.96 新 lint） - pages/admin/posts.rs: use_signal 冗余闭包（预存 lint，顺带修绿） - codemirror editor.ts: biome format 重排 setSchema 单行
This commit is contained in:
parent
9851f98dc0
commit
f3eb93f320
38
AGENTS.md
38
AGENTS.md
@ -75,6 +75,9 @@ RATE_LIMIT_COMMENT_PER_SEC=1 # comment posting
|
||||
RATE_LIMIT_COMMENT_BURST=5
|
||||
RATE_LIMIT_UNKNOWN_PER_SEC=30 # fallback bucket when real client IP can't be determined
|
||||
RATE_LIMIT_UNKNOWN_BURST=100
|
||||
RATE_LIMIT_CODE_EXEC_PER_SEC=1 # code runner per-IP burst (governor: integer only, no decimals)
|
||||
RATE_LIMIT_CODE_EXEC_BURST=3
|
||||
RATE_LIMIT_CODE_EXEC_DAILY=50 # code runner per-IP daily cap
|
||||
DB_POOL_SIZE=20 # database connection pool size
|
||||
MIGRATE_STARTUP_TIMEOUT_SECS=30 # how long startup waits for PostgreSQL before giving up
|
||||
STATEMENT_TIMEOUT_SECS=30 # per-query timeout; slow queries are canceled to protect the pool
|
||||
@ -82,6 +85,23 @@ SSR_CACHE_SECS=3600 # incremental SSR cache TTL (set 0 in dev)
|
||||
SYSINFO_SAMPLE_SECS=0.5 # sysinfo sampling interval in seconds, supports decimals
|
||||
```
|
||||
|
||||
Code Runner tuning (all optional, sane defaults):
|
||||
|
||||
```
|
||||
CODE_RUNNER_ALLOW_NETWORK=false # global network switch; AND-ed with per-language allow_network
|
||||
CODE_RUNNER_MAX_CONCURRENT=4 # tokio Semaphore for in-flight containers
|
||||
CODE_RUNNER_MAX_CPU_CORES=2.0 # upper clamp for cpu_cores (lower bound 0.1)
|
||||
CODE_RUNNER_MAX_MEMORY_MB=1024 # upper clamp for memory_mb (lower bound 16)
|
||||
CODE_RUNNER_MAX_TIMEOUT_SECS=30 # upper clamp for timeout_secs (lower bound 1)
|
||||
CODE_RUNNER_MAX_OUTPUT_BYTES=1048576 # hard cap on stdout+stderr captured
|
||||
CODE_RUNNER_MAX_SOURCE_BYTES=65536 # max source size accepted by StartExec
|
||||
CODE_RUNNER_QUEUE_TIMEOUT_SECS=30 # how long a task waits for a container slot before failing
|
||||
CODE_RUNNER_TASK_TTL_SECS=300 # DashMap task entry lifetime (gc_old_tasks)
|
||||
CODE_RUNNER_LANGUAGES=python,node # ops whitelist (must also exist in LANGUAGES registry)
|
||||
DOCKER_SOCKET_PATH=/var/run/docker.sock
|
||||
```
|
||||
|
||||
|
||||
Session / security tuning:
|
||||
|
||||
```
|
||||
@ -129,10 +149,11 @@ src/api/ — server functions + Axum handlers
|
||||
markdown.rs — Markdown→HTML rendering (pulldown-cmark + ammonia sanitization)
|
||||
image.rs — image serving with processing pipeline + disk+memory cache
|
||||
upload.rs — image upload, auto-converts to WebP
|
||||
rate_limit.rs — governor-based rate limiting (5 tiers: strict/upload/image/comment/unknown)
|
||||
rate_limit.rs — governor-based rate limiting (6 tiers: strict/upload/image/comment/unknown/code_exec)
|
||||
settings.rs — site settings server functions (trash retention, etc.)
|
||||
slug.rs — URL slug generation
|
||||
posts/ — CRUD server functions for blog posts
|
||||
code_runner/ — runnable code-block server functions + data structures (see Code Runner section)
|
||||
src/auth/ — password hashing (Argon2) + session token management
|
||||
src/bin/ — generate_highlight_css (build-time CSS generation)
|
||||
src/cache.rs — moka future-based caches for posts, tags, stats + cache_stats()
|
||||
@ -150,8 +171,23 @@ src/theme.rs — light/dark theme with SSR cookie + WASM localStorage
|
||||
src/webp.rs — zenwebp encode/decode (image crate has no WebP)
|
||||
src/tiptap_bridge.rs — wasm-bindgen bindings for Tiptap editor
|
||||
src/codemirror_bridge.rs — wasm-bindgen bindings for CodeMirror editor (mirrors tiptap_bridge)
|
||||
src/infra/ — Docker execution layer + runner config (server-only); see Code Runner section
|
||||
```
|
||||
|
||||
## Code Runner (` ```lang runnable ` code blocks)
|
||||
|
||||
Readers can execute fenced code blocks in isolated Docker containers; authors get a `/admin/runner` trial sandbox. Three-layer architecture, all Docker interaction gated behind `#[cfg(feature = "server")]`:
|
||||
|
||||
- **Execution layer** (`src/infra/docker.rs`, server-only): `bollard` client over Unix socket (`DOCKER_CLIENT` LazyLock). `run_in_container` creates a read-only-rootfs container (tmpfs `/code`,`/tmp`,`/run`; cpu/memory/pids/ulimits cap; `cap_drop=ALL`; `no-new-privileges`; non-root `1000:1000`; `network_mode` none/bridge), injects source via stdin, waits with timeout, captures stdout/stderr (truncated to `output_bytes`), inspects OOM, force-removes via `ContainerGuard` Drop. `src/infra/runner_config.rs` (`RUNNER_CONFIG`) reads all `CODE_RUNNER_*` env vars; `clamp_limits` AND-merges request overrides × language `allow_network` × global switch.
|
||||
- **API layer** (`src/api/code_runner/`): shared `ExecRequest`/`ExecResult`/`ExecStatus`/`ExecTask` structs (compile on both targets); `progress.rs` = DashMap task registry + `gc_old_tasks`; `languages.rs` = `LANGUAGES` registry + `parse_fence_info`; `execute.rs` = `StartExec`/`GetExecResult` server functions (double rate-limit → whitelist → size check → enqueue → `tokio::spawn` + `RUNNER_SEMAPHORE` concurrency cap → clamp → run_in_container). System errors are sanitized to 「系统暂时不可用」 for anonymous callers; full errors in server logs only.
|
||||
- **Markdown/render layer**: a fenced block ` ```python runnable {...overrides} ` renders to `<pre data-runnable="true" data-lang="python" data-overrides="..." data-source="...">` (sanitizer whitelists these 4 attrs on `<pre>`). `PostContent` (`src/components/post/post_content.rs`) splits `content_html` into `Html`/`Runnable` fragments so each runnable block renders as a real `<CodeRunner>` vdom element (no manual DOM mutation → no hydration conflict). `CodeRunner` component polls `GetExecResult` via WASM-friendly `sleep_ms`.
|
||||
|
||||
**Critical WASM-visibility rule**: `code_runner/execute.rs` is **not** cfg-gated (server functions must be visible to the client), but every server-only `use`/static inside it is individually `#[cfg(feature = "server")]`. The shared `use` of `ExecRequest`/`ExecTask` (used in signatures) stays ungated. `code_runner/languages.rs` and `code_runner/progress.rs` (pure server helpers) are wholly gated. Mirrors the `posts/` module convention.
|
||||
|
||||
**Governor 0.8 caveat**: `Quota::per_day` does not exist; `CODE_EXEC_DAILY_LIMITER` uses `Quota::with_period(24h).allow_burst(daily)`. `RATE_LIMIT_CODE_EXEC_PER_SEC` must be an integer (governor's `per_second` takes `NonZeroU32`; decimals in `.env` fall back to the default 1).
|
||||
|
||||
**Runner images** (`docker/`): `build-runners.sh` builds `yggdrasil-runner-base` → `yggdrasil-runner-python` → `yggdrasil-runner-node`; tags must match `LANGUAGES` image fields. Python image symlinks `python`→`python3` to match `run_cmd`. `runner.toml` files are image self-descriptions only — runtime config is the Rust `LANGUAGES` registry + `CODE_RUNNER_*` env, not parsed from toml.
|
||||
|
||||
## Frontend Lib Subprojects
|
||||
|
||||
Four Vite-built IIFE libraries under `libs/`, managed as a **pnpm workspace** (single `libs/pnpm-lock.yaml`, shared `libs/tsconfig.base.json` + `libs/biome.json`, hoisted devDeps). Built artifacts go to `public/<name>/` — **do not edit `public/<name>/` files; they are build artifacts**. Each `build` script is `tsc --noEmit && vite build` (type-check before bundle). Output is IIFE because Dioxus `[web.resource] script` injects bare `<script src>` without `type="module"` support. Registered globally in `Dioxus.toml` `script`/`style` arrays.
|
||||
|
||||
@ -78,9 +78,7 @@ export class CodeMirrorInstance {
|
||||
// vim 必须在 keymap 最前（@replit/codemirror-vim 仓库要求）
|
||||
this.vimCompartment.of(options.vim ? [vim()] : []),
|
||||
this.themeCompartment.of(themeExtension(theme)),
|
||||
this.languageCompartment.of(
|
||||
buildLanguageExtension(this.language, schema),
|
||||
),
|
||||
this.languageCompartment.of(buildLanguageExtension(this.language, schema)),
|
||||
EditorView.updateListener.of((v) => {
|
||||
if (v.docChanged) {
|
||||
options.onChange?.(this.view.state.doc.toString());
|
||||
@ -115,9 +113,7 @@ export class CodeMirrorInstance {
|
||||
setLanguage(lang: string): void {
|
||||
this.language = lang;
|
||||
this.view.dispatch({
|
||||
effects: this.languageCompartment.reconfigure(
|
||||
buildLanguageExtension(lang, this.schema),
|
||||
),
|
||||
effects: this.languageCompartment.reconfigure(buildLanguageExtension(lang, this.schema)),
|
||||
});
|
||||
}
|
||||
|
||||
@ -128,9 +124,7 @@ export class CodeMirrorInstance {
|
||||
// 仅当当前是 SQL 语言时才重配 extension（其它语言不消费 schema）。
|
||||
if ((this.language ?? '').toLowerCase() === 'sql' || !this.language) {
|
||||
this.view.dispatch({
|
||||
effects: this.languageCompartment.reconfigure(
|
||||
buildLanguageExtension('sql', schema),
|
||||
),
|
||||
effects: this.languageCompartment.reconfigure(buildLanguageExtension('sql', schema)),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@ -12,9 +12,8 @@ use std::sync::LazyLock;
|
||||
|
||||
use crate::infra::runner_config::{ResourceLimits, RUNNER_CONFIG};
|
||||
|
||||
/// 单个语言的运行定义。
|
||||
/// 单个语言的运行定义。语言名即 [`LANGUAGES`] 的 key，不再冗余存字段。
|
||||
pub struct LanguageDef {
|
||||
pub name: String,
|
||||
/// 容器镜像（task 12 的 docker build 产出，如 `yggdrasil-runner-python:latest`）。
|
||||
pub image: String,
|
||||
/// 容器内执行命令（源码会注入到 `/code/main.{ext}`）。
|
||||
@ -33,7 +32,6 @@ pub static LANGUAGES: LazyLock<HashMap<String, LanguageDef>> = LazyLock::new(||
|
||||
m.insert(
|
||||
"python".to_string(),
|
||||
LanguageDef {
|
||||
name: "python".to_string(),
|
||||
image: "yggdrasil-runner-python:latest".to_string(),
|
||||
run_cmd: "python /code/main.py".to_string(),
|
||||
extension: "py".to_string(),
|
||||
@ -51,7 +49,6 @@ pub static LANGUAGES: LazyLock<HashMap<String, LanguageDef>> = LazyLock::new(||
|
||||
m.insert(
|
||||
"node".to_string(),
|
||||
LanguageDef {
|
||||
name: "node".to_string(),
|
||||
image: "yggdrasil-runner-node:latest".to_string(),
|
||||
run_cmd: "node /code/main.js".to_string(),
|
||||
extension: "js".to_string(),
|
||||
|
||||
@ -90,9 +90,9 @@ pub fn render_markdown_enhanced(md: &str) -> RenderedContent {
|
||||
let mut in_heading = false;
|
||||
let mut in_codeblock = false;
|
||||
let mut code_lang: Option<String> = None;
|
||||
/// 可运行代码块的 (lang, html-escaped overrides JSON)。
|
||||
/// 为 None 表示普通代码块。原始源码在 End 处从 code_buffer 转义后存入 data-source，
|
||||
/// 供阅读器无损提取（避免从高亮 HTML 反解）。
|
||||
// 可运行代码块的 (lang, html-escaped overrides JSON)。
|
||||
// 为 None 表示普通代码块。原始源码在 End 处从 code_buffer 转义后存入 data-source，
|
||||
// 供阅读器无损提取（避免从高亮 HTML 反解）。
|
||||
let mut code_runnable: Option<(String, String)> = None;
|
||||
let mut code_buffer = String::new();
|
||||
let mut non_heading_events: Vec<Event> = Vec::new();
|
||||
@ -157,9 +157,7 @@ pub fn render_markdown_enhanced(md: &str) -> RenderedContent {
|
||||
};
|
||||
// 解析围栏 info：识别 `runnable` 标记与可选 ResourceLimits JSON 覆盖。
|
||||
// 仅在「标记为 runnable 且语言受支持」时挂 data-*，供阅读器扫描挂载运行器。
|
||||
code_runnable = code_lang
|
||||
.as_deref()
|
||||
.map(|info| {
|
||||
code_runnable = code_lang.as_deref().and_then(|info| {
|
||||
let (lang, runnable, overrides) =
|
||||
crate::api::code_runner::languages::parse_fence_info(info);
|
||||
if runnable && crate::api::code_runner::languages::is_supported_lang(&lang) {
|
||||
@ -172,8 +170,7 @@ pub fn render_markdown_enhanced(md: &str) -> RenderedContent {
|
||||
} else {
|
||||
None
|
||||
}
|
||||
})
|
||||
.flatten();
|
||||
});
|
||||
code_buffer.clear();
|
||||
}
|
||||
Event::Text(text) if in_codeblock => {
|
||||
|
||||
@ -29,10 +29,10 @@ pub fn CodeRunner(
|
||||
overrides: Option<ResourceLimits>,
|
||||
) -> Element {
|
||||
let mut running = use_signal(|| false);
|
||||
let mut stage = use_signal(|| String::new());
|
||||
let mut output = use_signal(|| String::new());
|
||||
let mut exit_info = use_signal(|| String::new());
|
||||
let mut error_msg = use_signal(|| String::new());
|
||||
let mut stage = use_signal(String::new);
|
||||
let mut output = use_signal(String::new);
|
||||
let mut exit_info = use_signal(String::new);
|
||||
let mut error_msg = use_signal(String::new);
|
||||
|
||||
// 为每个实例生成稳定的容器 id（CodeMirror 容器，由调用方在 WASM 端挂载）。
|
||||
// use_id 在 Dioxus 0.7 提供 scope 内稳定的 id；退回 use_hook 保证只算一次。
|
||||
|
||||
@ -118,9 +118,9 @@ pub async fn run_in_container(
|
||||
};
|
||||
|
||||
// Start container
|
||||
if let Err(e) = docker.start_container(&container_id, None::<StartContainerOptions<String>>).await {
|
||||
return Err(e);
|
||||
}
|
||||
docker
|
||||
.start_container(&container_id, None::<StartContainerOptions<String>>)
|
||||
.await?;
|
||||
|
||||
// Write source code to stdin and drop/close the writer
|
||||
use tokio::io::AsyncWriteExt;
|
||||
@ -130,7 +130,7 @@ pub async fn run_in_container(
|
||||
let _ = writer.shutdown().await;
|
||||
};
|
||||
|
||||
if let Err(_) = timeout(Duration::from_secs(5), write_fut).await {
|
||||
if timeout(Duration::from_secs(5), write_fut).await.is_err() {
|
||||
return Err(bollard::errors::Error::IOError {
|
||||
err: std::io::Error::new(std::io::ErrorKind::TimedOut, "Writing to stdin timed out")
|
||||
});
|
||||
@ -366,12 +366,10 @@ mod tests {
|
||||
let mut leaked = Vec::new();
|
||||
for c in after {
|
||||
let id = c.id.unwrap();
|
||||
if !before_ids.contains(&id) {
|
||||
if c.image.as_deref() == Some("alpine:latest") {
|
||||
if !before_ids.contains(&id) && c.image.as_deref() == Some("alpine:latest") {
|
||||
leaked.push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let leaked_count = leaked.len();
|
||||
for id in leaked {
|
||||
|
||||
@ -56,7 +56,7 @@ pub static RUNNER_CONFIG: LazyLock<RunnerConfig> = LazyLock::new(|| {
|
||||
|
||||
#[cfg(feature = "server")]
|
||||
pub fn clamp_limits(merged: ResourceLimits, lang_allows_network: bool) -> ResourceLimits {
|
||||
clamp_limits_impl(merged, lang_allows_network, &*RUNNER_CONFIG)
|
||||
clamp_limits_impl(merged, lang_allows_network, &RUNNER_CONFIG)
|
||||
}
|
||||
|
||||
#[cfg(feature = "server")]
|
||||
@ -102,7 +102,7 @@ mod tests {
|
||||
assert!(clamped.memory_mb <= 1024);
|
||||
assert!(clamped.timeout_secs <= 30);
|
||||
assert!(clamped.output_bytes <= 1048576);
|
||||
assert_eq!(clamped.allow_network, false);
|
||||
assert!(!clamped.allow_network);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@ -132,7 +132,7 @@ mod tests {
|
||||
assert_eq!(clamped.memory_mb, 8);
|
||||
assert_eq!(clamped.timeout_secs, 0);
|
||||
assert_eq!(clamped.output_bytes, 50);
|
||||
assert_eq!(clamped.allow_network, true);
|
||||
assert!(clamped.allow_network);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@ -60,10 +60,10 @@ pub fn PostsPage(page: i32) -> Element {
|
||||
// 删除中 / 重建中文章 ID 集合：均由本组件持有（业务逻辑不归 hook 管）。
|
||||
// 改为非乐观删除后行会保留至请求完成，可并发点多个删除，故用 HashSet
|
||||
// 与 rebuilding 同形，按行通过 contains 判断 loading 态。
|
||||
let mut deleting = use_signal(|| std::collections::HashSet::<i32>::new());
|
||||
let mut deleting = use_signal(std::collections::HashSet::<i32>::new);
|
||||
// 重建中文章 ID 集合：支持多篇文章并发重建（行不会随点击消失，单值会被后点
|
||||
// 的覆盖先点的，故用 HashSet），按行通过 contains 判断 loading 态。
|
||||
let mut rebuilding = use_signal(|| std::collections::HashSet::<i32>::new());
|
||||
let mut rebuilding = use_signal(std::collections::HashSet::<i32>::new);
|
||||
// 重建缓存的状态由本组件持有并下发给 RebuildCacheBar：结果消息也在本组件
|
||||
// 渲染（header 与表格之间的独立行），既不撑高 header 触发 items-center 重排，
|
||||
// 也不脱离文档流溢进表格。rebuilding 仅按钮态用，不在此渲染。
|
||||
|
||||
@ -31,8 +31,8 @@ pub fn Runner() -> Element {
|
||||
let mut lang = use_signal(|| "python".to_string());
|
||||
// 语言切换时刷新示例源码（首次进入也有默认值）。
|
||||
let mut source = use_signal(|| default_source("python"));
|
||||
let mut overrides_json = use_signal(|| String::new());
|
||||
let mut override_error = use_signal(|| String::new());
|
||||
let mut overrides_json = use_signal(String::new);
|
||||
let mut override_error = use_signal(String::new);
|
||||
|
||||
let current_source = source();
|
||||
let current_lang = lang();
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user