refactor(file): add ripgrep search service (#22295)
This commit is contained in:
@@ -13,7 +13,7 @@
|
|||||||
|
|
||||||
Use these rules when writing or migrating Effect code.
|
Use these rules when writing or migrating Effect code.
|
||||||
|
|
||||||
See `specs/effect-migration.md` for the compact pattern reference and examples.
|
See `specs/effect/migration.md` for the compact pattern reference and examples.
|
||||||
|
|
||||||
## Core
|
## Core
|
||||||
|
|
||||||
@@ -51,7 +51,7 @@ See `specs/effect-migration.md` for the compact pattern reference and examples.
|
|||||||
|
|
||||||
## Effect.cached for deduplication
|
## Effect.cached for deduplication
|
||||||
|
|
||||||
Use `Effect.cached` when multiple concurrent callers should share a single in-flight computation rather than storing `Fiber | undefined` or `Promise | undefined` manually. See `specs/effect-migration.md` for the full pattern.
|
Use `Effect.cached` when multiple concurrent callers should share a single in-flight computation rather than storing `Fiber | undefined` or `Promise | undefined` manually. See `specs/effect/migration.md` for the full pattern.
|
||||||
|
|
||||||
## Instance.bind — ALS for native callbacks
|
## Instance.bind — ALS for native callbacks
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,137 @@
|
|||||||
|
# HttpApi migration
|
||||||
|
|
||||||
|
Practical notes for an eventual migration of `packages/opencode` server routes from the current Hono handlers to Effect `HttpApi`, either as a full replacement or as a parallel surface.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Use Effect `HttpApi` where it gives us a better typed contract for:
|
||||||
|
|
||||||
|
- route definition
|
||||||
|
- request decoding and validation
|
||||||
|
- typed success and error responses
|
||||||
|
- OpenAPI generation
|
||||||
|
- handler composition inside Effect
|
||||||
|
|
||||||
|
This should be treated as a later-stage HTTP boundary migration, not a prerequisite for ongoing service, route-handler, or schema work.
|
||||||
|
|
||||||
|
## Core model
|
||||||
|
|
||||||
|
`HttpApi` is definition-first.
|
||||||
|
|
||||||
|
- `HttpApi` is the root API
|
||||||
|
- `HttpApiGroup` groups related endpoints
|
||||||
|
- `HttpApiEndpoint` defines a single route and its request / response schemas
|
||||||
|
- handlers are implemented separately from the contract
|
||||||
|
|
||||||
|
This is a better fit once route inputs and outputs are already moving toward Effect Schema-first models.
|
||||||
|
|
||||||
|
## Why it is relevant here
|
||||||
|
|
||||||
|
The current route-effectification work is already pushing handlers toward:
|
||||||
|
|
||||||
|
- one `AppRuntime.runPromise(Effect.gen(...))` body
|
||||||
|
- yielding services from context
|
||||||
|
- using typed Effect errors instead of Promise wrappers
|
||||||
|
|
||||||
|
That work is a good prerequisite for `HttpApi`. Once the handler body is already a composed Effect, the remaining migration is mostly about replacing the Hono route declaration and validator layer.
|
||||||
|
|
||||||
|
## What HttpApi gives us
|
||||||
|
|
||||||
|
### Contracts
|
||||||
|
|
||||||
|
Request params, query, payload, success payloads, and typed error payloads are declared in one place using Effect Schema.
|
||||||
|
|
||||||
|
### Validation and decoding
|
||||||
|
|
||||||
|
Incoming data is decoded through Effect Schema instead of hand-maintained Zod validators per route.
|
||||||
|
|
||||||
|
### OpenAPI
|
||||||
|
|
||||||
|
`HttpApi` can derive OpenAPI from the API definition, which overlaps with the current `describeRoute(...)` and `resolver(...)` pattern.
|
||||||
|
|
||||||
|
### Typed errors
|
||||||
|
|
||||||
|
`Schema.TaggedErrorClass` maps naturally to endpoint error contracts.
|
||||||
|
|
||||||
|
## Likely fit for opencode
|
||||||
|
|
||||||
|
Best fit first:
|
||||||
|
|
||||||
|
- JSON request / response endpoints
|
||||||
|
- route groups that already mostly delegate into services
|
||||||
|
- endpoints whose request and response models can be defined with Effect Schema
|
||||||
|
|
||||||
|
Harder / later fit:
|
||||||
|
|
||||||
|
- SSE endpoints
|
||||||
|
- websocket endpoints
|
||||||
|
- streaming handlers
|
||||||
|
- routes with heavy Hono-specific middleware assumptions
|
||||||
|
|
||||||
|
## Current blockers and gaps
|
||||||
|
|
||||||
|
### Schema split
|
||||||
|
|
||||||
|
Many route boundaries still use Zod-first validators. That does not block all experimentation, but full `HttpApi` adoption is easier after the domain and boundary types are more consistently Schema-first with `.zod` compatibility only where needed.
|
||||||
|
|
||||||
|
### Mixed handler styles
|
||||||
|
|
||||||
|
Many current `server/instance/*.ts` handlers still call async facades directly. Migrating those to composed `Effect.gen(...)` handlers is the low-risk step to do first.
|
||||||
|
|
||||||
|
### Non-JSON routes
|
||||||
|
|
||||||
|
The server currently includes SSE, websocket, and streaming-style endpoints. Those should not be the first `HttpApi` targets.
|
||||||
|
|
||||||
|
### Existing Hono integration
|
||||||
|
|
||||||
|
The current server composition, middleware, and docs flow are Hono-centered today. That suggests a parallel or incremental adoption plan is safer than a flag day rewrite.
|
||||||
|
|
||||||
|
## Recommended strategy
|
||||||
|
|
||||||
|
### 1. Finish the prerequisites first
|
||||||
|
|
||||||
|
- continue route-handler effectification in `server/instance/*.ts`
|
||||||
|
- continue schema migration toward Effect Schema-first DTOs and errors
|
||||||
|
- keep removing service facades
|
||||||
|
|
||||||
|
### 2. Start with one parallel group
|
||||||
|
|
||||||
|
Introduce one small `HttpApi` group for plain JSON endpoints only. Good initial candidates are the least stateful endpoints in:
|
||||||
|
|
||||||
|
- `server/instance/question.ts`
|
||||||
|
- `server/instance/provider.ts`
|
||||||
|
- `server/instance/permission.ts`
|
||||||
|
|
||||||
|
Avoid `session.ts`, SSE, websocket, and TUI-facing routes first.
|
||||||
|
|
||||||
|
### 3. Reuse existing services
|
||||||
|
|
||||||
|
Do not re-architect business logic during the HTTP migration. `HttpApi` handlers should call the same Effect services already used by the Hono handlers.
|
||||||
|
|
||||||
|
### 4. Run in parallel before replacing
|
||||||
|
|
||||||
|
Prefer mounting an experimental `HttpApi` surface alongside the existing Hono routes first. That lowers migration risk and lets us compare:
|
||||||
|
|
||||||
|
- handler ergonomics
|
||||||
|
- OpenAPI output
|
||||||
|
- auth and middleware integration
|
||||||
|
- test ergonomics
|
||||||
|
|
||||||
|
### 5. Migrate JSON route groups gradually
|
||||||
|
|
||||||
|
If the parallel slice works well, migrate additional JSON route groups one at a time. Leave streaming-style endpoints on Hono until there is a clear reason to move them.
|
||||||
|
|
||||||
|
## Proposed first steps
|
||||||
|
|
||||||
|
- [ ] add one small spike that defines an `HttpApi` group for a simple JSON route set
|
||||||
|
- [ ] use Effect Schema request / response types for that slice
|
||||||
|
- [ ] keep the underlying service calls identical to the current handlers
|
||||||
|
- [ ] compare generated OpenAPI against the current Hono/OpenAPI setup
|
||||||
|
- [ ] document how auth, instance lookup, and error mapping would compose in the new stack
|
||||||
|
- [ ] decide after the spike whether `HttpApi` should stay parallel, replace only some groups, or become the long-term default
|
||||||
|
|
||||||
|
## Rule of thumb
|
||||||
|
|
||||||
|
Do not start with the hardest route file.
|
||||||
|
|
||||||
|
If `HttpApi` is adopted here, it should arrive after the handler body is already Effect-native and after the relevant request / response models have moved to Effect Schema.
|
||||||
+7
-90
@@ -230,55 +230,9 @@ Still open at the service-shape level:
|
|||||||
- [ ] `SyncEvent` — `sync/index.ts` (deferred pending sync with James)
|
- [ ] `SyncEvent` — `sync/index.ts` (deferred pending sync with James)
|
||||||
- [ ] `Workspace` — `control-plane/workspace.ts` (deferred pending sync with James)
|
- [ ] `Workspace` — `control-plane/workspace.ts` (deferred pending sync with James)
|
||||||
|
|
||||||
## Tool interface → Effect
|
## Tool migration
|
||||||
|
|
||||||
`Tool.Def.execute` and `Tool.Info.init` already return `Effect` on this branch, and the current tools in `src/tool/*.ts` have been migrated to the Effect-native `Tool.define(...)` shape.
|
Tool-specific migration guidance and checklist live in `tools.md`.
|
||||||
|
|
||||||
The remaining work here is follow-on cleanup rather than the top-level tool interface migration:
|
|
||||||
|
|
||||||
1. Remove internal `Effect.promise(...)` bridges where practical
|
|
||||||
2. Keep replacing raw platform helpers with Effect services inside tool bodies
|
|
||||||
3. Update remaining callers and tests to prefer `yield* info.init()` / `Tool.init(...)` over older Promise-oriented patterns
|
|
||||||
|
|
||||||
### Tool migration details
|
|
||||||
|
|
||||||
With `Tool.Info.init()` now effectful, use this transitional pattern for migrated tools that still need Promise-based boundaries internally:
|
|
||||||
|
|
||||||
- `Tool.defineEffect(...)` should `yield*` the services the tool depends on and close over them in the returned tool definition.
|
|
||||||
- Keep the bridge at the Promise boundary only inside the tool body when required by external APIs. Do not return Promise-based init callbacks from `Tool.define()`.
|
|
||||||
- If a tool starts requiring new services, wire them into `ToolRegistry.defaultLayer` so production callers resolve the same dependencies as tests.
|
|
||||||
|
|
||||||
Tool tests should use the existing Effect helpers in `packages/opencode/test/lib/effect.ts`:
|
|
||||||
|
|
||||||
- Use `testEffect(...)` / `it.live(...)` instead of creating fake local wrappers around effectful tools.
|
|
||||||
- Yield the real tool export, then initialize it: `const info = yield* ReadTool`, `const tool = yield* info.init()`.
|
|
||||||
- Run tests inside a real instance with `provideTmpdirInstance(...)` or `provideInstance(tmpdirScoped(...))` so instance-scoped services resolve exactly as they do in production.
|
|
||||||
|
|
||||||
This keeps migrated tool tests aligned with the production service graph today, and makes the eventual `Tool.Info` → `Effect` cleanup mostly mechanical later.
|
|
||||||
|
|
||||||
Individual tools, ordered by value:
|
|
||||||
|
|
||||||
- [x] `apply_patch.ts` — HIGH: multi-step orchestration, error accumulation, Bus events
|
|
||||||
- [x] `bash.ts` — HIGH: shell orchestration, quoting, timeout handling, output capture
|
|
||||||
- [x] `read.ts` — HIGH: effectful interface migrated; still has raw fs/readline internals tracked below
|
|
||||||
- [x] `edit.ts` — HIGH: multi-step diff/format/publish pipeline, FileWatcher lock
|
|
||||||
- [x] `grep.ts` — MEDIUM: spawns ripgrep → ChildProcessSpawner, timeout handling
|
|
||||||
- [x] `write.ts` — MEDIUM: permission checks, diagnostics polling, Bus events
|
|
||||||
- [x] `codesearch.ts` — MEDIUM: HTTP + SSE + manual timeout → HttpClient + Effect.timeout
|
|
||||||
- [x] `webfetch.ts` — MEDIUM: fetch with UA retry, size limits → HttpClient
|
|
||||||
- [x] `websearch.ts` — MEDIUM: MCP over HTTP → HttpClient
|
|
||||||
- [x] `task.ts` — MEDIUM: task state management
|
|
||||||
- [x] `ls.ts` — MEDIUM: bounded directory listing over ripgrep-backed traversal
|
|
||||||
- [x] `multiedit.ts` — MEDIUM: sequential edit orchestration over `edit.ts`
|
|
||||||
- [x] `glob.ts` — LOW: simple async generator
|
|
||||||
- [x] `lsp.ts` — LOW: dispatch switch over LSP operations
|
|
||||||
- [x] `question.ts` — LOW: prompt wrapper
|
|
||||||
- [x] `skill.ts` — LOW: skill tool adapter
|
|
||||||
- [x] `todo.ts` — LOW: todo persistence wrapper
|
|
||||||
- [x] `invalid.ts` — LOW: invalid-tool fallback
|
|
||||||
- [x] `plan.ts` — LOW: plan file operations
|
|
||||||
|
|
||||||
`batch.ts` was removed from `src/tool/` and is no longer tracked here.
|
|
||||||
|
|
||||||
## Effect service adoption in already-migrated code
|
## Effect service adoption in already-migrated code
|
||||||
|
|
||||||
@@ -298,11 +252,7 @@ Some already-effectified areas still use raw `Filesystem.*` or `Process.spawn` i
|
|||||||
|
|
||||||
`util/filesystem.ts` is still used widely across `src/`, and raw `fs` / `fs/promises` imports still exist in multiple tooling and infrastructure files. As services and tools are effectified, they should switch from `Filesystem.*` to yielding `AppFileSystem.Service` where possible — this should happen naturally during each migration, not as a separate sweep.
|
`util/filesystem.ts` is still used widely across `src/`, and raw `fs` / `fs/promises` imports still exist in multiple tooling and infrastructure files. As services and tools are effectified, they should switch from `Filesystem.*` to yielding `AppFileSystem.Service` where possible — this should happen naturally during each migration, not as a separate sweep.
|
||||||
|
|
||||||
Current raw fs users that will convert during tool migration:
|
Tool-specific filesystem cleanup notes live in `tools.md`.
|
||||||
|
|
||||||
- `tool/read.ts` — fs.createReadStream, readline
|
|
||||||
- `file/ripgrep.ts` — fs/promises
|
|
||||||
- `patch/index.ts` — fs, fs/promises
|
|
||||||
|
|
||||||
## Primitives & utilities
|
## Primitives & utilities
|
||||||
|
|
||||||
@@ -344,47 +294,14 @@ For each service, the migration is roughly:
|
|||||||
- `ShareNext` — migrated 2026-04-11. Swapped remaining async callers to `AppRuntime.runPromise(ShareNext.Service.use(...))`, removed the `makeRuntime(...)` facade, and kept instance bootstrap on the shared app runtime.
|
- `ShareNext` — migrated 2026-04-11. Swapped remaining async callers to `AppRuntime.runPromise(ShareNext.Service.use(...))`, removed the `makeRuntime(...)` facade, and kept instance bootstrap on the shared app runtime.
|
||||||
- `SessionTodo` — migrated 2026-04-10. Already matched the target service shape in `session/todo.ts`: single namespace, traced Effect methods, and no `makeRuntime(...)` facade remained; checklist updated to reflect the completed migration.
|
- `SessionTodo` — migrated 2026-04-10. Already matched the target service shape in `session/todo.ts`: single namespace, traced Effect methods, and no `makeRuntime(...)` facade remained; checklist updated to reflect the completed migration.
|
||||||
- `Storage` — migrated 2026-04-10. One production caller (`Session.diff`) and all storage.test.ts tests converted to effectful style. Facades and `makeRuntime` removed.
|
- `Storage` — migrated 2026-04-10. One production caller (`Session.diff`) and all storage.test.ts tests converted to effectful style. Facades and `makeRuntime` removed.
|
||||||
- `SessionRunState` — migrated 2026-04-11. Single caller in `server/routes/session.ts` converted; facade removed.
|
- `SessionRunState` — migrated 2026-04-11. Single caller in `server/instance/session.ts` converted; facade removed.
|
||||||
- `Account` — migrated 2026-04-11. Callers in `server/routes/experimental.ts` and `cli/cmd/account.ts` converted; facade removed.
|
- `Account` — migrated 2026-04-11. Callers in `server/instance/experimental.ts` and `cli/cmd/account.ts` converted; facade removed.
|
||||||
- `Instruction` — migrated 2026-04-11. Test-only callers converted; facade removed.
|
- `Instruction` — migrated 2026-04-11. Test-only callers converted; facade removed.
|
||||||
- `FileTime` — migrated 2026-04-11. Test-only callers converted; facade removed.
|
- `FileTime` — migrated 2026-04-11. Test-only callers converted; facade removed.
|
||||||
- `FileWatcher` — migrated 2026-04-11. Callers in `project/bootstrap.ts` and test converted; facade removed.
|
- `FileWatcher` — migrated 2026-04-11. Callers in `project/bootstrap.ts` and test converted; facade removed.
|
||||||
- `Question` — migrated 2026-04-11. Callers in `server/routes/question.ts` and test converted; facade removed.
|
- `Question` — migrated 2026-04-11. Callers in `server/instance/question.ts` and test converted; facade removed.
|
||||||
- `Truncate` — migrated 2026-04-11. Caller in `tool/tool.ts` and test converted; facade removed.
|
- `Truncate` — migrated 2026-04-11. Caller in `tool/tool.ts` and test converted; facade removed.
|
||||||
|
|
||||||
## Route handler effectification
|
## Route handler effectification
|
||||||
|
|
||||||
Route handlers should wrap their entire body in a single `AppRuntime.runPromise(Effect.gen(...))` call, yielding services from context rather than calling facades one-by-one. This eliminates multiple `runPromise` round-trips and lets handlers compose naturally.
|
Route-handler migration guidance and checklist live in `routes.md`.
|
||||||
|
|
||||||
```ts
|
|
||||||
// Before — one facade call per service
|
|
||||||
;async (c) => {
|
|
||||||
await SessionRunState.assertNotBusy(id)
|
|
||||||
await Session.removeMessage({ sessionID: id, messageID })
|
|
||||||
return c.json(true)
|
|
||||||
}
|
|
||||||
|
|
||||||
// After — one Effect.gen, yield services from context
|
|
||||||
;async (c) => {
|
|
||||||
await AppRuntime.runPromise(
|
|
||||||
Effect.gen(function* () {
|
|
||||||
const state = yield* SessionRunState.Service
|
|
||||||
const session = yield* Session.Service
|
|
||||||
yield* state.assertNotBusy(id)
|
|
||||||
yield* session.removeMessage({ sessionID: id, messageID })
|
|
||||||
}),
|
|
||||||
)
|
|
||||||
return c.json(true)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
When migrating, always use `{ concurrency: "unbounded" }` with `Effect.all` — route handlers should run independent service calls in parallel, not sequentially.
|
|
||||||
|
|
||||||
Route files to convert (each handler that calls facades should be wrapped):
|
|
||||||
|
|
||||||
- [ ] `server/routes/session.ts` — heaviest; uses Session, SessionPrompt, SessionRevert, SessionCompaction, SessionShare, SessionSummary, SessionRunState, Agent, Permission, Bus
|
|
||||||
- [ ] `server/routes/global.ts` — uses Config, Project, Provider, Vcs, Snapshot, Agent
|
|
||||||
- [ ] `server/routes/provider.ts` — uses Provider, Auth, Config
|
|
||||||
- [ ] `server/routes/question.ts` — uses Question
|
|
||||||
- [ ] `server/routes/pty.ts` — uses Pty
|
|
||||||
- [ ] `server/routes/experimental.ts` — uses Account, ToolRegistry, Agent, MCP, Config
|
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
# Route handler effectification
|
||||||
|
|
||||||
|
Practical reference for converting server route handlers in `packages/opencode` to a single `AppRuntime.runPromise(Effect.gen(...))` body.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Route handlers should wrap their entire body in a single `AppRuntime.runPromise(Effect.gen(...))` call, yielding services from context rather than calling facades one-by-one.
|
||||||
|
|
||||||
|
This eliminates multiple `runPromise` round-trips and lets handlers compose naturally.
|
||||||
|
|
||||||
|
```ts
|
||||||
|
// Before - one facade call per service
|
||||||
|
;async (c) => {
|
||||||
|
await SessionRunState.assertNotBusy(id)
|
||||||
|
await Session.removeMessage({ sessionID: id, messageID })
|
||||||
|
return c.json(true)
|
||||||
|
}
|
||||||
|
|
||||||
|
// After - one Effect.gen, yield services from context
|
||||||
|
;async (c) => {
|
||||||
|
await AppRuntime.runPromise(
|
||||||
|
Effect.gen(function* () {
|
||||||
|
const state = yield* SessionRunState.Service
|
||||||
|
const session = yield* Session.Service
|
||||||
|
yield* state.assertNotBusy(id)
|
||||||
|
yield* session.removeMessage({ sessionID: id, messageID })
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
return c.json(true)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
- Wrap the whole handler body in one `AppRuntime.runPromise(Effect.gen(...))` call when the handler is service-heavy.
|
||||||
|
- Yield services from context instead of calling async facades repeatedly.
|
||||||
|
- When independent service calls can run in parallel, use `Effect.all(..., { concurrency: "unbounded" })`.
|
||||||
|
- Prefer one composed Effect body over multiple separate `runPromise(...)` calls in the same handler.
|
||||||
|
|
||||||
|
## Current route files
|
||||||
|
|
||||||
|
Current instance route files live under `src/server/instance`, not `server/routes`.
|
||||||
|
|
||||||
|
The main migration targets are:
|
||||||
|
|
||||||
|
- [ ] `server/instance/session.ts` — heaviest; still has many direct facade calls for Session, SessionPrompt, SessionRevert, SessionCompaction, SessionShare, SessionSummary, Agent, Bus
|
||||||
|
- [ ] `server/instance/global.ts` — still has direct facade calls for Config and instance lifecycle actions
|
||||||
|
- [ ] `server/instance/provider.ts` — still has direct facade calls for Config and Provider
|
||||||
|
- [ ] `server/instance/question.ts` — partially converted; still worth tracking here until it consistently uses the composed style
|
||||||
|
- [ ] `server/instance/pty.ts` — still calls Pty facades directly
|
||||||
|
- [ ] `server/instance/experimental.ts` — mixed state; some handlers are already composed, others still use facades
|
||||||
|
|
||||||
|
Additional route files that still participate in the migration:
|
||||||
|
|
||||||
|
- [ ] `server/instance/index.ts` — Vcs, Agent, Skill, LSP, Format
|
||||||
|
- [ ] `server/instance/file.ts` — Ripgrep, File, LSP
|
||||||
|
- [ ] `server/instance/mcp.ts` — MCP facade-heavy
|
||||||
|
- [ ] `server/instance/permission.ts` — Permission
|
||||||
|
- [ ] `server/instance/workspace.ts` — Workspace
|
||||||
|
- [ ] `server/instance/tui.ts` — Bus and Session
|
||||||
|
- [ ] `server/instance/middleware.ts` — Session and Workspace lookups
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Some handlers already use `AppRuntime.runPromise(Effect.gen(...))` in isolated places. Keep pushing those files toward one consistent style.
|
||||||
|
- Route conversion is closely tied to facade removal. As services lose `makeRuntime`-backed async exports, route handlers should switch to yielding the service directly.
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
# Schema migration
|
||||||
|
|
||||||
|
Practical reference for migrating data types in `packages/opencode` from Zod-first definitions to Effect Schema with Zod compatibility shims.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Use Effect Schema as the source of truth for domain models, IDs, inputs, outputs, and typed errors.
|
||||||
|
|
||||||
|
Keep Zod available at existing HTTP, tool, and compatibility boundaries by exposing a `.zod` field derived from the Effect schema.
|
||||||
|
|
||||||
|
## Preferred shapes
|
||||||
|
|
||||||
|
### Data objects
|
||||||
|
|
||||||
|
Use `Schema.Class` for structured data.
|
||||||
|
|
||||||
|
```ts
|
||||||
|
export class Info extends Schema.Class<Info>("Foo.Info")({
|
||||||
|
id: FooID,
|
||||||
|
name: Schema.String,
|
||||||
|
enabled: Schema.Boolean,
|
||||||
|
}) {
|
||||||
|
static readonly zod = zod(Info)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
If the class cannot reference itself cleanly during initialization, use the existing two-step pattern:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const _Info = Schema.Struct({
|
||||||
|
id: FooID,
|
||||||
|
name: Schema.String,
|
||||||
|
})
|
||||||
|
|
||||||
|
export const Info = Object.assign(_Info, {
|
||||||
|
zod: zod(_Info),
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
### Errors
|
||||||
|
|
||||||
|
Use `Schema.TaggedErrorClass` for domain errors.
|
||||||
|
|
||||||
|
```ts
|
||||||
|
export class NotFoundError extends Schema.TaggedErrorClass<NotFoundError>()("FooNotFoundError", {
|
||||||
|
id: FooID,
|
||||||
|
}) {}
|
||||||
|
```
|
||||||
|
|
||||||
|
### IDs and branded leaf types
|
||||||
|
|
||||||
|
Keep branded/schema-backed IDs as Effect schemas and expose `static readonly zod` for compatibility when callers still expect Zod.
|
||||||
|
|
||||||
|
## Compatibility rule
|
||||||
|
|
||||||
|
During migration, route validators, tool parameters, and any existing Zod-based boundary should consume the derived `.zod` schema instead of maintaining a second hand-written Zod schema.
|
||||||
|
|
||||||
|
The default should be:
|
||||||
|
|
||||||
|
- Effect Schema owns the type
|
||||||
|
- `.zod` exists only as a compatibility surface
|
||||||
|
- new domain models should not start Zod-first unless there is a concrete boundary-specific need
|
||||||
|
|
||||||
|
## When Zod can stay
|
||||||
|
|
||||||
|
It is fine to keep a Zod-native schema temporarily when:
|
||||||
|
|
||||||
|
- the type is only used at an HTTP or tool boundary
|
||||||
|
- the validator depends on Zod-only transforms or behavior not yet covered by `zod()`
|
||||||
|
- the migration would force unrelated churn across a large call graph
|
||||||
|
|
||||||
|
When this happens, prefer leaving a short note or TODO rather than silently creating a parallel schema source of truth.
|
||||||
|
|
||||||
|
## Ordering
|
||||||
|
|
||||||
|
Migrate in this order:
|
||||||
|
|
||||||
|
1. Shared leaf models and `schema.ts` files
|
||||||
|
2. Exported `Info`, `Input`, `Output`, and DTO types
|
||||||
|
3. Tagged domain errors
|
||||||
|
4. Service-local internal models
|
||||||
|
5. Route and tool boundary validators that can switch to `.zod`
|
||||||
|
|
||||||
|
This keeps shared types canonical first and makes boundary updates mostly mechanical.
|
||||||
|
|
||||||
|
## Checklist
|
||||||
|
|
||||||
|
- [ ] Shared `schema.ts` leaf models are Effect Schema-first
|
||||||
|
- [ ] Exported `Info` / `Input` / `Output` types use `Schema.Class` where appropriate
|
||||||
|
- [ ] Domain errors use `Schema.TaggedErrorClass`
|
||||||
|
- [ ] Migrated types expose `.zod` for back compatibility
|
||||||
|
- [ ] Route and tool validators consume derived `.zod` instead of duplicate Zod definitions
|
||||||
|
- [ ] New domain models default to Effect Schema first
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Use `@/util/effect-zod` for all Schema -> Zod conversion.
|
||||||
|
- Prefer one canonical schema definition. Avoid maintaining parallel Zod and Effect definitions for the same domain type.
|
||||||
|
- Keep the migration incremental. Converting the domain model first is more valuable than converting every boundary in the same change.
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
# Tool migration
|
||||||
|
|
||||||
|
Practical reference for the current tool-migration state in `packages/opencode`.
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
`Tool.Def.execute` and `Tool.Info.init` already return `Effect` on this branch, and the built-in tool surface is now largely on the target shape.
|
||||||
|
|
||||||
|
The current exported tools in `src/tool` all use `Tool.define(...)` with Effect-based initialization, and nearly all of them already build their tool body with `Effect.gen(...)` and `Effect.fn(...)`.
|
||||||
|
|
||||||
|
So the remaining work is no longer "convert tools to Effect at all". The remaining work is mostly:
|
||||||
|
|
||||||
|
1. remove Promise and raw platform bridges inside individual tool bodies
|
||||||
|
2. swap tool internals to Effect-native services like `AppFileSystem`, `HttpClient`, and `ChildProcessSpawner`
|
||||||
|
3. keep tests and callers aligned with `yield* info.init()` and real service graphs
|
||||||
|
|
||||||
|
## Current shape
|
||||||
|
|
||||||
|
`Tool.define(...)` is already the Effect-native helper here.
|
||||||
|
|
||||||
|
- `init` is an `Effect`
|
||||||
|
- `info.init()` returns an `Effect`
|
||||||
|
- `execute(...)` returns an `Effect`
|
||||||
|
|
||||||
|
That means a tool does not need a separate `Tool.defineEffect(...)` helper to count as migrated. A tool is effectively migrated when its init and execute path stay Effect-native, even if some internals still bridge to Promise-based or raw APIs.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
Tool tests should use the existing Effect helpers in `packages/opencode/test/lib/effect.ts`:
|
||||||
|
|
||||||
|
- Use `testEffect(...)` / `it.live(...)` instead of creating fake local wrappers around effectful tools.
|
||||||
|
- Yield the real tool export, then initialize it: `const info = yield* ReadTool`, `const tool = yield* info.init()`.
|
||||||
|
- Run tests inside a real instance with `provideTmpdirInstance(...)` or `provideInstance(tmpdirScoped(...))` so instance-scoped services resolve exactly as they do in production.
|
||||||
|
|
||||||
|
This keeps tool tests aligned with the production service graph and makes follow-up cleanup mostly mechanical.
|
||||||
|
|
||||||
|
## Exported tools
|
||||||
|
|
||||||
|
These exported tool definitions already exist in `src/tool` and are on the current Effect-native `Tool.define(...)` path:
|
||||||
|
|
||||||
|
- [x] `apply_patch.ts`
|
||||||
|
- [x] `bash.ts`
|
||||||
|
- [x] `codesearch.ts`
|
||||||
|
- [x] `edit.ts`
|
||||||
|
- [x] `glob.ts`
|
||||||
|
- [x] `grep.ts`
|
||||||
|
- [x] `invalid.ts`
|
||||||
|
- [x] `ls.ts`
|
||||||
|
- [x] `lsp.ts`
|
||||||
|
- [x] `multiedit.ts`
|
||||||
|
- [x] `plan.ts`
|
||||||
|
- [x] `question.ts`
|
||||||
|
- [x] `read.ts`
|
||||||
|
- [x] `skill.ts`
|
||||||
|
- [x] `task.ts`
|
||||||
|
- [x] `todo.ts`
|
||||||
|
- [x] `webfetch.ts`
|
||||||
|
- [x] `websearch.ts`
|
||||||
|
- [x] `write.ts`
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
|
||||||
|
- `batch.ts` is no longer a current tool file and should not be tracked here.
|
||||||
|
- `truncate.ts` is an Effect service used by tools, not a tool definition itself.
|
||||||
|
- `mcp-exa.ts`, `external-directory.ts`, and `schema.ts` are support modules, not standalone tool definitions.
|
||||||
|
|
||||||
|
## Follow-up cleanup
|
||||||
|
|
||||||
|
Most exported tools are already on the intended Effect-native shape. The remaining cleanup is narrower than the old checklist implied.
|
||||||
|
|
||||||
|
Current spot cleanups worth tracking:
|
||||||
|
|
||||||
|
- [ ] `read.ts` — still bridges to Node stream / `readline` helpers and Promise-based binary detection
|
||||||
|
- [ ] `bash.ts` — already uses Effect child-process primitives; only keep tracking shell-specific platform bridges and parser/loading details as they come up
|
||||||
|
- [ ] `webfetch.ts` — already uses `HttpClient`; remaining work is limited to smaller boundary helpers like HTML text extraction
|
||||||
|
- [ ] `file/ripgrep.ts` — adjacent to tool migration; still has raw fs/process usage that affects `grep.ts` and `ls.ts`
|
||||||
|
- [ ] `patch/index.ts` — adjacent to tool migration; still has raw fs usage behind patch application
|
||||||
|
|
||||||
|
Notable items that are already effectively on the target path and do not need separate migration bullets right now:
|
||||||
|
|
||||||
|
- `apply_patch.ts`
|
||||||
|
- `grep.ts`
|
||||||
|
- `write.ts`
|
||||||
|
- `codesearch.ts`
|
||||||
|
- `websearch.ts`
|
||||||
|
- `ls.ts`
|
||||||
|
- `multiedit.ts`
|
||||||
|
- `edit.ts`
|
||||||
|
|
||||||
|
## Filesystem notes
|
||||||
|
|
||||||
|
Current raw fs users that still appear relevant here:
|
||||||
|
|
||||||
|
- `tool/read.ts` — `fs.createReadStream`, `readline`
|
||||||
|
- `file/ripgrep.ts` — `fs/promises`
|
||||||
|
- `patch/index.ts` — `fs`, `fs/promises`
|
||||||
@@ -1,4 +1,5 @@
|
|||||||
import { EOL } from "os"
|
import { EOL } from "os"
|
||||||
|
import { AppRuntime } from "../../../effect/app-runtime"
|
||||||
import { Ripgrep } from "../../../file/ripgrep"
|
import { Ripgrep } from "../../../file/ripgrep"
|
||||||
import { Instance } from "../../../project/instance"
|
import { Instance } from "../../../project/instance"
|
||||||
import { bootstrap } from "../../bootstrap"
|
import { bootstrap } from "../../bootstrap"
|
||||||
@@ -76,12 +77,18 @@ const SearchCommand = cmd({
|
|||||||
description: "Limit number of results",
|
description: "Limit number of results",
|
||||||
}),
|
}),
|
||||||
async handler(args) {
|
async handler(args) {
|
||||||
const results = await Ripgrep.search({
|
await bootstrap(process.cwd(), async () => {
|
||||||
cwd: process.cwd(),
|
const results = await AppRuntime.runPromise(
|
||||||
pattern: args.pattern,
|
Ripgrep.Service.use((svc) =>
|
||||||
glob: args.glob as string[] | undefined,
|
svc.search({
|
||||||
limit: args.limit,
|
cwd: Instance.directory,
|
||||||
|
pattern: args.pattern,
|
||||||
|
glob: args.glob as string[] | undefined,
|
||||||
|
limit: args.limit,
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
process.stdout.write(JSON.stringify(results.items, null, 2) + EOL)
|
||||||
})
|
})
|
||||||
process.stdout.write(JSON.stringify(results, null, 2) + EOL)
|
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -96,6 +96,7 @@ export namespace Ripgrep {
|
|||||||
|
|
||||||
export type Result = z.infer<typeof Result>
|
export type Result = z.infer<typeof Result>
|
||||||
export type Match = z.infer<typeof Match>
|
export type Match = z.infer<typeof Match>
|
||||||
|
export type Item = Match["data"]
|
||||||
export type Begin = z.infer<typeof Begin>
|
export type Begin = z.infer<typeof Begin>
|
||||||
export type End = z.infer<typeof End>
|
export type End = z.infer<typeof End>
|
||||||
export type Summary = z.infer<typeof Summary>
|
export type Summary = z.infer<typeof Summary>
|
||||||
@@ -289,6 +290,13 @@ export namespace Ripgrep {
|
|||||||
follow?: boolean
|
follow?: boolean
|
||||||
maxDepth?: number
|
maxDepth?: number
|
||||||
}) => Stream.Stream<string, PlatformError>
|
}) => Stream.Stream<string, PlatformError>
|
||||||
|
readonly search: (input: {
|
||||||
|
cwd: string
|
||||||
|
pattern: string
|
||||||
|
glob?: string[]
|
||||||
|
limit?: number
|
||||||
|
follow?: boolean
|
||||||
|
}) => Effect.Effect<{ items: Item[]; partial: boolean }, PlatformError | Error>
|
||||||
}
|
}
|
||||||
|
|
||||||
export class Service extends Context.Service<Service, Interface>()("@opencode/Ripgrep") {}
|
export class Service extends Context.Service<Service, Interface>()("@opencode/Ripgrep") {}
|
||||||
@@ -298,6 +306,32 @@ export namespace Ripgrep {
|
|||||||
Effect.gen(function* () {
|
Effect.gen(function* () {
|
||||||
const spawner = yield* ChildProcessSpawner
|
const spawner = yield* ChildProcessSpawner
|
||||||
const afs = yield* AppFileSystem.Service
|
const afs = yield* AppFileSystem.Service
|
||||||
|
const bin = Effect.fn("Ripgrep.path")(function* () {
|
||||||
|
return yield* Effect.promise(() => filepath())
|
||||||
|
})
|
||||||
|
const args = Effect.fn("Ripgrep.args")(function* (input: {
|
||||||
|
mode: "files" | "search"
|
||||||
|
glob?: string[]
|
||||||
|
hidden?: boolean
|
||||||
|
follow?: boolean
|
||||||
|
maxDepth?: number
|
||||||
|
limit?: number
|
||||||
|
pattern?: string
|
||||||
|
}) {
|
||||||
|
const out = [yield* bin(), input.mode === "search" ? "--json" : "--files", "--glob=!.git/*"]
|
||||||
|
if (input.follow) out.push("--follow")
|
||||||
|
if (input.hidden !== false) out.push("--hidden")
|
||||||
|
if (input.maxDepth !== undefined) out.push(`--max-depth=${input.maxDepth}`)
|
||||||
|
if (input.glob) {
|
||||||
|
for (const g of input.glob) {
|
||||||
|
out.push(`--glob=${g}`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (input.limit) out.push(`--max-count=${input.limit}`)
|
||||||
|
if (input.mode === "search") out.push("--no-messages")
|
||||||
|
if (input.pattern) out.push("--", input.pattern)
|
||||||
|
return out
|
||||||
|
})
|
||||||
|
|
||||||
const files = Effect.fn("Ripgrep.files")(function* (input: {
|
const files = Effect.fn("Ripgrep.files")(function* (input: {
|
||||||
cwd: string
|
cwd: string
|
||||||
@@ -306,7 +340,7 @@ export namespace Ripgrep {
|
|||||||
follow?: boolean
|
follow?: boolean
|
||||||
maxDepth?: number
|
maxDepth?: number
|
||||||
}) {
|
}) {
|
||||||
const rgPath = yield* Effect.promise(() => filepath())
|
const rgPath = yield* bin()
|
||||||
const isDir = yield* afs.isDir(input.cwd)
|
const isDir = yield* afs.isDir(input.cwd)
|
||||||
if (!isDir) {
|
if (!isDir) {
|
||||||
return yield* Effect.die(
|
return yield* Effect.die(
|
||||||
@@ -318,23 +352,76 @@ export namespace Ripgrep {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
const args = [rgPath, "--files", "--glob=!.git/*"]
|
const cmd = yield* args({
|
||||||
if (input.follow) args.push("--follow")
|
mode: "files",
|
||||||
if (input.hidden !== false) args.push("--hidden")
|
glob: input.glob,
|
||||||
if (input.maxDepth !== undefined) args.push(`--max-depth=${input.maxDepth}`)
|
hidden: input.hidden,
|
||||||
if (input.glob) {
|
follow: input.follow,
|
||||||
for (const g of input.glob) {
|
maxDepth: input.maxDepth,
|
||||||
args.push(`--glob=${g}`)
|
})
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return spawner
|
return spawner
|
||||||
.streamLines(ChildProcess.make(args[0], args.slice(1), { cwd: input.cwd }))
|
.streamLines(ChildProcess.make(cmd[0], cmd.slice(1), { cwd: input.cwd }))
|
||||||
.pipe(Stream.filter((line: string) => line.length > 0))
|
.pipe(Stream.filter((line: string) => line.length > 0))
|
||||||
})
|
})
|
||||||
|
|
||||||
|
const search = Effect.fn("Ripgrep.search")(function* (input: {
|
||||||
|
cwd: string
|
||||||
|
pattern: string
|
||||||
|
glob?: string[]
|
||||||
|
limit?: number
|
||||||
|
follow?: boolean
|
||||||
|
}) {
|
||||||
|
return yield* Effect.scoped(
|
||||||
|
Effect.gen(function* () {
|
||||||
|
const cmd = yield* args({
|
||||||
|
mode: "search",
|
||||||
|
glob: input.glob,
|
||||||
|
follow: input.follow,
|
||||||
|
limit: input.limit,
|
||||||
|
pattern: input.pattern,
|
||||||
|
})
|
||||||
|
|
||||||
|
const handle = yield* spawner.spawn(
|
||||||
|
ChildProcess.make(cmd[0], cmd.slice(1), {
|
||||||
|
cwd: input.cwd,
|
||||||
|
stdin: "ignore",
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
|
||||||
|
const [stdout, stderr, code] = yield* Effect.all(
|
||||||
|
[
|
||||||
|
Stream.mkString(Stream.decodeText(handle.stdout)),
|
||||||
|
Stream.mkString(Stream.decodeText(handle.stderr)),
|
||||||
|
handle.exitCode,
|
||||||
|
],
|
||||||
|
{ concurrency: "unbounded" },
|
||||||
|
)
|
||||||
|
|
||||||
|
if (code !== 0 && code !== 1 && code !== 2) {
|
||||||
|
return yield* Effect.fail(new Error(`ripgrep failed: ${stderr}`))
|
||||||
|
}
|
||||||
|
|
||||||
|
const items = stdout
|
||||||
|
.trim()
|
||||||
|
.split(/\r?\n/)
|
||||||
|
.filter(Boolean)
|
||||||
|
.map((line) => JSON.parse(line))
|
||||||
|
.map((parsed) => Result.parse(parsed))
|
||||||
|
.filter((row): row is Match => row.type === "match")
|
||||||
|
.map((row) => row.data)
|
||||||
|
|
||||||
|
return {
|
||||||
|
items,
|
||||||
|
partial: code === 2,
|
||||||
|
}
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
return Service.of({
|
return Service.of({
|
||||||
files: (input) => Stream.unwrap(files(input)),
|
files: (input) => Stream.unwrap(files(input)),
|
||||||
|
search,
|
||||||
})
|
})
|
||||||
}),
|
}),
|
||||||
)
|
)
|
||||||
@@ -401,46 +488,4 @@ export namespace Ripgrep {
|
|||||||
|
|
||||||
return lines.join("\n")
|
return lines.join("\n")
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function search(input: {
|
|
||||||
cwd: string
|
|
||||||
pattern: string
|
|
||||||
glob?: string[]
|
|
||||||
limit?: number
|
|
||||||
follow?: boolean
|
|
||||||
}) {
|
|
||||||
const args = [`${await filepath()}`, "--json", "--hidden", "--glob=!.git/*"]
|
|
||||||
if (input.follow) args.push("--follow")
|
|
||||||
|
|
||||||
if (input.glob) {
|
|
||||||
for (const g of input.glob) {
|
|
||||||
args.push(`--glob=${g}`)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if (input.limit) {
|
|
||||||
args.push(`--max-count=${input.limit}`)
|
|
||||||
}
|
|
||||||
|
|
||||||
args.push("--")
|
|
||||||
args.push(input.pattern)
|
|
||||||
|
|
||||||
const result = await Process.text(args, {
|
|
||||||
cwd: input.cwd,
|
|
||||||
nothrow: true,
|
|
||||||
})
|
|
||||||
if (result.code !== 0) {
|
|
||||||
return []
|
|
||||||
}
|
|
||||||
|
|
||||||
// Handle both Unix (\n) and Windows (\r\n) line endings
|
|
||||||
const lines = result.text.trim().split(/\r?\n/).filter(Boolean)
|
|
||||||
// Parse JSON lines from ripgrep output
|
|
||||||
|
|
||||||
return lines
|
|
||||||
.map((line) => JSON.parse(line))
|
|
||||||
.map((parsed) => Result.parse(parsed))
|
|
||||||
.filter((r) => r.type === "match")
|
|
||||||
.map((r) => r.data)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { Hono } from "hono"
|
import { Hono } from "hono"
|
||||||
import { describeRoute, validator, resolver } from "hono-openapi"
|
import { describeRoute, validator, resolver } from "hono-openapi"
|
||||||
import z from "zod"
|
import z from "zod"
|
||||||
|
import { AppRuntime } from "../../effect/app-runtime"
|
||||||
import { File } from "../../file"
|
import { File } from "../../file"
|
||||||
import { Ripgrep } from "../../file/ripgrep"
|
import { Ripgrep } from "../../file/ripgrep"
|
||||||
import { LSP } from "../../lsp"
|
import { LSP } from "../../lsp"
|
||||||
@@ -34,12 +35,10 @@ export const FileRoutes = lazy(() =>
|
|||||||
),
|
),
|
||||||
async (c) => {
|
async (c) => {
|
||||||
const pattern = c.req.valid("query").pattern
|
const pattern = c.req.valid("query").pattern
|
||||||
const result = await Ripgrep.search({
|
const result = await AppRuntime.runPromise(
|
||||||
cwd: Instance.directory,
|
Ripgrep.Service.use((svc) => svc.search({ cwd: Instance.directory, pattern, limit: 10 })),
|
||||||
pattern,
|
)
|
||||||
limit: 10,
|
return c.json(result.items)
|
||||||
})
|
|
||||||
return c.json(result)
|
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
.get(
|
.get(
|
||||||
|
|||||||
@@ -1,11 +1,8 @@
|
|||||||
import z from "zod"
|
import z from "zod"
|
||||||
import { Effect } from "effect"
|
import { Effect, Option } from "effect"
|
||||||
import * as Stream from "effect/Stream"
|
|
||||||
import { Tool } from "./tool"
|
import { Tool } from "./tool"
|
||||||
import { Filesystem } from "../util/filesystem"
|
|
||||||
import { Ripgrep } from "../file/ripgrep"
|
import { Ripgrep } from "../file/ripgrep"
|
||||||
import { ChildProcess } from "effect/unstable/process"
|
import { AppFileSystem } from "../filesystem"
|
||||||
import { ChildProcessSpawner } from "effect/unstable/process/ChildProcessSpawner"
|
|
||||||
|
|
||||||
import DESCRIPTION from "./grep.txt"
|
import DESCRIPTION from "./grep.txt"
|
||||||
import { Instance } from "../project/instance"
|
import { Instance } from "../project/instance"
|
||||||
@@ -17,7 +14,8 @@ const MAX_LINE_LENGTH = 2000
|
|||||||
export const GrepTool = Tool.define(
|
export const GrepTool = Tool.define(
|
||||||
"grep",
|
"grep",
|
||||||
Effect.gen(function* () {
|
Effect.gen(function* () {
|
||||||
const spawner = yield* ChildProcessSpawner
|
const fs = yield* AppFileSystem.Service
|
||||||
|
const rg = yield* Ripgrep.Service
|
||||||
|
|
||||||
return {
|
return {
|
||||||
description: DESCRIPTION,
|
description: DESCRIPTION,
|
||||||
@@ -28,6 +26,11 @@ export const GrepTool = Tool.define(
|
|||||||
}),
|
}),
|
||||||
execute: (params: { pattern: string; path?: string; include?: string }, ctx: Tool.Context) =>
|
execute: (params: { pattern: string; path?: string; include?: string }, ctx: Tool.Context) =>
|
||||||
Effect.gen(function* () {
|
Effect.gen(function* () {
|
||||||
|
const empty = {
|
||||||
|
title: params.pattern,
|
||||||
|
metadata: { matches: 0, truncated: false },
|
||||||
|
output: "No files found",
|
||||||
|
}
|
||||||
if (!params.pattern) {
|
if (!params.pattern) {
|
||||||
throw new Error("pattern is required")
|
throw new Error("pattern is required")
|
||||||
}
|
}
|
||||||
@@ -43,92 +46,58 @@ export const GrepTool = Tool.define(
|
|||||||
},
|
},
|
||||||
})
|
})
|
||||||
|
|
||||||
let searchPath = params.path ?? Instance.directory
|
const searchPath = AppFileSystem.resolve(
|
||||||
searchPath = path.isAbsolute(searchPath) ? searchPath : path.resolve(Instance.directory, searchPath)
|
path.isAbsolute(params.path ?? Instance.directory)
|
||||||
|
? (params.path ?? Instance.directory)
|
||||||
|
: path.join(Instance.directory, params.path ?? "."),
|
||||||
|
)
|
||||||
yield* assertExternalDirectoryEffect(ctx, searchPath, { kind: "directory" })
|
yield* assertExternalDirectoryEffect(ctx, searchPath, { kind: "directory" })
|
||||||
|
|
||||||
const rgPath = yield* Effect.promise(() => Ripgrep.filepath())
|
const result = yield* rg.search({
|
||||||
const args = ["-nH", "--hidden", "--no-messages", "--field-match-separator=|", "--regexp", params.pattern]
|
cwd: searchPath,
|
||||||
if (params.include) {
|
pattern: params.pattern,
|
||||||
args.push("--glob", params.include)
|
glob: params.include ? [params.include] : undefined,
|
||||||
}
|
})
|
||||||
args.push(searchPath)
|
|
||||||
|
|
||||||
const result = yield* Effect.scoped(
|
if (result.items.length === 0) return empty
|
||||||
Effect.gen(function* () {
|
|
||||||
const handle = yield* spawner.spawn(
|
|
||||||
ChildProcess.make(rgPath, args, {
|
|
||||||
stdin: "ignore",
|
|
||||||
}),
|
|
||||||
)
|
|
||||||
|
|
||||||
const [output, errorOutput] = yield* Effect.all(
|
const rows = result.items.map((item) => ({
|
||||||
[Stream.mkString(Stream.decodeText(handle.stdout)), Stream.mkString(Stream.decodeText(handle.stderr))],
|
path: AppFileSystem.resolve(
|
||||||
{ concurrency: 2 },
|
path.isAbsolute(item.path.text) ? item.path.text : path.join(searchPath, item.path.text),
|
||||||
)
|
),
|
||||||
|
line: item.line_number,
|
||||||
const exitCode = yield* handle.exitCode
|
text: item.lines.text,
|
||||||
|
}))
|
||||||
return { output, errorOutput, exitCode }
|
const times = new Map(
|
||||||
}),
|
(yield* Effect.forEach(
|
||||||
|
[...new Set(rows.map((row) => row.path))],
|
||||||
|
Effect.fnUntraced(function* (file) {
|
||||||
|
const info = yield* fs.stat(file).pipe(Effect.catch(() => Effect.succeed(undefined)))
|
||||||
|
if (!info || info.type === "Directory") return undefined
|
||||||
|
return [
|
||||||
|
file,
|
||||||
|
info.mtime.pipe(
|
||||||
|
Option.map((time) => time.getTime()),
|
||||||
|
Option.getOrElse(() => 0),
|
||||||
|
) ?? 0,
|
||||||
|
] as const
|
||||||
|
}),
|
||||||
|
{ concurrency: 16 },
|
||||||
|
)).filter((entry): entry is readonly [string, number] => Boolean(entry)),
|
||||||
)
|
)
|
||||||
|
const matches = rows.flatMap((row) => {
|
||||||
|
const mtime = times.get(row.path)
|
||||||
|
if (mtime === undefined) return []
|
||||||
|
return [{ ...row, mtime }]
|
||||||
|
})
|
||||||
|
|
||||||
const { output, errorOutput, exitCode } = result
|
matches.sort((a, b) => b.mtime - a.mtime)
|
||||||
|
|
||||||
// Exit codes: 0 = matches found, 1 = no matches, 2 = errors (but may still have matches)
|
|
||||||
// With --no-messages, we suppress error output but still get exit code 2 for broken symlinks etc.
|
|
||||||
// Only fail if exit code is 2 AND no output was produced
|
|
||||||
if (exitCode === 1 || (exitCode === 2 && !output.trim())) {
|
|
||||||
return {
|
|
||||||
title: params.pattern,
|
|
||||||
metadata: { matches: 0, truncated: false },
|
|
||||||
output: "No files found",
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if (exitCode !== 0 && exitCode !== 2) {
|
|
||||||
throw new Error(`ripgrep failed: ${errorOutput}`)
|
|
||||||
}
|
|
||||||
|
|
||||||
const hasErrors = exitCode === 2
|
|
||||||
|
|
||||||
// Handle both Unix (\n) and Windows (\r\n) line endings
|
|
||||||
const lines = output.trim().split(/\r?\n/)
|
|
||||||
const matches = []
|
|
||||||
|
|
||||||
for (const line of lines) {
|
|
||||||
if (!line) continue
|
|
||||||
|
|
||||||
const [filePath, lineNumStr, ...lineTextParts] = line.split("|")
|
|
||||||
if (!filePath || !lineNumStr || lineTextParts.length === 0) continue
|
|
||||||
|
|
||||||
const lineNum = parseInt(lineNumStr, 10)
|
|
||||||
const lineText = lineTextParts.join("|")
|
|
||||||
|
|
||||||
const stats = Filesystem.stat(filePath)
|
|
||||||
if (!stats) continue
|
|
||||||
|
|
||||||
matches.push({
|
|
||||||
path: filePath,
|
|
||||||
modTime: stats.mtime.getTime(),
|
|
||||||
lineNum,
|
|
||||||
lineText,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
matches.sort((a, b) => b.modTime - a.modTime)
|
|
||||||
|
|
||||||
const limit = 100
|
const limit = 100
|
||||||
const truncated = matches.length > limit
|
const truncated = matches.length > limit
|
||||||
const finalMatches = truncated ? matches.slice(0, limit) : matches
|
const finalMatches = truncated ? matches.slice(0, limit) : matches
|
||||||
|
|
||||||
if (finalMatches.length === 0) {
|
if (finalMatches.length === 0) return empty
|
||||||
return {
|
|
||||||
title: params.pattern,
|
|
||||||
metadata: { matches: 0, truncated: false },
|
|
||||||
output: "No files found",
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
const totalMatches = matches.length
|
const totalMatches = matches.length
|
||||||
const outputLines = [`Found ${totalMatches} matches${truncated ? ` (showing first ${limit})` : ""}`]
|
const outputLines = [`Found ${totalMatches} matches${truncated ? ` (showing first ${limit})` : ""}`]
|
||||||
@@ -143,10 +112,8 @@ export const GrepTool = Tool.define(
|
|||||||
outputLines.push(`${match.path}:`)
|
outputLines.push(`${match.path}:`)
|
||||||
}
|
}
|
||||||
const truncatedLineText =
|
const truncatedLineText =
|
||||||
match.lineText.length > MAX_LINE_LENGTH
|
match.text.length > MAX_LINE_LENGTH ? match.text.substring(0, MAX_LINE_LENGTH) + "..." : match.text
|
||||||
? match.lineText.substring(0, MAX_LINE_LENGTH) + "..."
|
outputLines.push(` Line ${match.line}: ${truncatedLineText}`)
|
||||||
: match.lineText
|
|
||||||
outputLines.push(` Line ${match.lineNum}: ${truncatedLineText}`)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if (truncated) {
|
if (truncated) {
|
||||||
@@ -156,7 +123,7 @@ export const GrepTool = Tool.define(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if (hasErrors) {
|
if (result.partial) {
|
||||||
outputLines.push("")
|
outputLines.push("")
|
||||||
outputLines.push("(Some paths were inaccessible and skipped)")
|
outputLines.push("(Some paths were inaccessible and skipped)")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -38,7 +38,9 @@ describe("file.ripgrep", () => {
|
|||||||
expect(hasVisible).toBe(true)
|
expect(hasVisible).toBe(true)
|
||||||
expect(hasHidden).toBe(false)
|
expect(hasHidden).toBe(false)
|
||||||
})
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe("Ripgrep.Service", () => {
|
||||||
test("search returns empty when nothing matches", async () => {
|
test("search returns empty when nothing matches", async () => {
|
||||||
await using tmp = await tmpdir({
|
await using tmp = await tmpdir({
|
||||||
init: async (dir) => {
|
init: async (dir) => {
|
||||||
@@ -46,16 +48,34 @@ describe("file.ripgrep", () => {
|
|||||||
},
|
},
|
||||||
})
|
})
|
||||||
|
|
||||||
const hits = await Ripgrep.search({
|
const result = await Effect.gen(function* () {
|
||||||
cwd: tmp.path,
|
const rg = yield* Ripgrep.Service
|
||||||
pattern: "needle",
|
return yield* rg.search({ cwd: tmp.path, pattern: "needle" })
|
||||||
|
}).pipe(Effect.provide(Ripgrep.defaultLayer), Effect.runPromise)
|
||||||
|
|
||||||
|
expect(result.partial).toBe(false)
|
||||||
|
expect(result.items).toEqual([])
|
||||||
|
})
|
||||||
|
|
||||||
|
test("search returns matched rows", async () => {
|
||||||
|
await using tmp = await tmpdir({
|
||||||
|
init: async (dir) => {
|
||||||
|
await Bun.write(path.join(dir, "match.ts"), "const value = 'needle'\n")
|
||||||
|
await Bun.write(path.join(dir, "skip.txt"), "const value = 'other'\n")
|
||||||
|
},
|
||||||
})
|
})
|
||||||
|
|
||||||
expect(hits).toEqual([])
|
const result = await Effect.gen(function* () {
|
||||||
})
|
const rg = yield* Ripgrep.Service
|
||||||
})
|
return yield* rg.search({ cwd: tmp.path, pattern: "needle", glob: ["*.ts"] })
|
||||||
|
}).pipe(Effect.provide(Ripgrep.defaultLayer), Effect.runPromise)
|
||||||
|
|
||||||
|
expect(result.partial).toBe(false)
|
||||||
|
expect(result.items).toHaveLength(1)
|
||||||
|
expect(result.items[0]?.path.text).toContain("match.ts")
|
||||||
|
expect(result.items[0]?.lines.text).toContain("needle")
|
||||||
|
})
|
||||||
|
|
||||||
describe("Ripgrep.Service", () => {
|
|
||||||
test("files returns stream of filenames", async () => {
|
test("files returns stream of filenames", async () => {
|
||||||
await using tmp = await tmpdir({
|
await using tmp = await tmpdir({
|
||||||
init: async (dir) => {
|
init: async (dir) => {
|
||||||
|
|||||||
@@ -1,22 +1,26 @@
|
|||||||
import { describe, expect, test } from "bun:test"
|
import { describe, expect } from "bun:test"
|
||||||
import path from "path"
|
import path from "path"
|
||||||
import { Effect, Layer, ManagedRuntime } from "effect"
|
import { Effect, Layer } from "effect"
|
||||||
import { GrepTool } from "../../src/tool/grep"
|
import { GrepTool } from "../../src/tool/grep"
|
||||||
import { Instance } from "../../src/project/instance"
|
import { provideInstance, provideTmpdirInstance } from "../fixture/fixture"
|
||||||
import { tmpdir } from "../fixture/fixture"
|
|
||||||
import { SessionID, MessageID } from "../../src/session/schema"
|
import { SessionID, MessageID } from "../../src/session/schema"
|
||||||
import * as CrossSpawnSpawner from "../../src/effect/cross-spawn-spawner"
|
import * as CrossSpawnSpawner from "../../src/effect/cross-spawn-spawner"
|
||||||
import { Truncate } from "../../src/tool/truncate"
|
import { Truncate } from "../../src/tool/truncate"
|
||||||
import { Agent } from "../../src/agent/agent"
|
import { Agent } from "../../src/agent/agent"
|
||||||
|
import { Ripgrep } from "../../src/file/ripgrep"
|
||||||
|
import { AppFileSystem } from "../../src/filesystem"
|
||||||
|
import { testEffect } from "../lib/effect"
|
||||||
|
|
||||||
const runtime = ManagedRuntime.make(
|
const it = testEffect(
|
||||||
Layer.mergeAll(CrossSpawnSpawner.defaultLayer, Truncate.defaultLayer, Agent.defaultLayer),
|
Layer.mergeAll(
|
||||||
|
CrossSpawnSpawner.defaultLayer,
|
||||||
|
AppFileSystem.defaultLayer,
|
||||||
|
Ripgrep.defaultLayer,
|
||||||
|
Truncate.defaultLayer,
|
||||||
|
Agent.defaultLayer,
|
||||||
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
function initGrep() {
|
|
||||||
return runtime.runPromise(GrepTool.pipe(Effect.flatMap((info) => info.init())))
|
|
||||||
}
|
|
||||||
|
|
||||||
const ctx = {
|
const ctx = {
|
||||||
sessionID: SessionID.make("ses_test"),
|
sessionID: SessionID.make("ses_test"),
|
||||||
messageID: MessageID.make(""),
|
messageID: MessageID.make(""),
|
||||||
@@ -31,99 +35,59 @@ const ctx = {
|
|||||||
const projectRoot = path.join(__dirname, "../..")
|
const projectRoot = path.join(__dirname, "../..")
|
||||||
|
|
||||||
describe("tool.grep", () => {
|
describe("tool.grep", () => {
|
||||||
test("basic search", async () => {
|
it.live("basic search", () =>
|
||||||
await Instance.provide({
|
Effect.gen(function* () {
|
||||||
directory: projectRoot,
|
const info = yield* GrepTool
|
||||||
fn: async () => {
|
const grep = yield* info.init()
|
||||||
const grep = await initGrep()
|
const result = yield* provideInstance(projectRoot)(
|
||||||
const result = await Effect.runPromise(
|
grep.execute(
|
||||||
grep.execute(
|
{
|
||||||
{
|
pattern: "export",
|
||||||
pattern: "export",
|
path: path.join(projectRoot, "src/tool"),
|
||||||
path: path.join(projectRoot, "src/tool"),
|
include: "*.ts",
|
||||||
include: "*.ts",
|
},
|
||||||
},
|
ctx,
|
||||||
ctx,
|
),
|
||||||
),
|
)
|
||||||
)
|
expect(result.metadata.matches).toBeGreaterThan(0)
|
||||||
expect(result.metadata.matches).toBeGreaterThan(0)
|
expect(result.output).toContain("Found")
|
||||||
expect(result.output).toContain("Found")
|
}),
|
||||||
},
|
)
|
||||||
})
|
|
||||||
})
|
|
||||||
|
|
||||||
test("no matches returns correct output", async () => {
|
it.live("no matches returns correct output", () =>
|
||||||
await using tmp = await tmpdir({
|
provideTmpdirInstance((dir) =>
|
||||||
init: async (dir) => {
|
Effect.gen(function* () {
|
||||||
await Bun.write(path.join(dir, "test.txt"), "hello world")
|
yield* Effect.promise(() => Bun.write(path.join(dir, "test.txt"), "hello world"))
|
||||||
},
|
const info = yield* GrepTool
|
||||||
})
|
const grep = yield* info.init()
|
||||||
await Instance.provide({
|
const result = yield* grep.execute(
|
||||||
directory: tmp.path,
|
{
|
||||||
fn: async () => {
|
pattern: "xyznonexistentpatternxyz123",
|
||||||
const grep = await initGrep()
|
path: dir,
|
||||||
const result = await Effect.runPromise(
|
},
|
||||||
grep.execute(
|
ctx,
|
||||||
{
|
|
||||||
pattern: "xyznonexistentpatternxyz123",
|
|
||||||
path: tmp.path,
|
|
||||||
},
|
|
||||||
ctx,
|
|
||||||
),
|
|
||||||
)
|
)
|
||||||
expect(result.metadata.matches).toBe(0)
|
expect(result.metadata.matches).toBe(0)
|
||||||
expect(result.output).toBe("No files found")
|
expect(result.output).toBe("No files found")
|
||||||
},
|
}),
|
||||||
})
|
),
|
||||||
})
|
)
|
||||||
|
|
||||||
test("handles CRLF line endings in output", async () => {
|
it.live("finds matches in tmp instance", () =>
|
||||||
// This test verifies the regex split handles both \n and \r\n
|
provideTmpdirInstance((dir) =>
|
||||||
await using tmp = await tmpdir({
|
Effect.gen(function* () {
|
||||||
init: async (dir) => {
|
yield* Effect.promise(() => Bun.write(path.join(dir, "test.txt"), "line1\nline2\nline3"))
|
||||||
// Create a test file with content
|
const info = yield* GrepTool
|
||||||
await Bun.write(path.join(dir, "test.txt"), "line1\nline2\nline3")
|
const grep = yield* info.init()
|
||||||
},
|
const result = yield* grep.execute(
|
||||||
})
|
{
|
||||||
await Instance.provide({
|
pattern: "line",
|
||||||
directory: tmp.path,
|
path: dir,
|
||||||
fn: async () => {
|
},
|
||||||
const grep = await initGrep()
|
ctx,
|
||||||
const result = await Effect.runPromise(
|
|
||||||
grep.execute(
|
|
||||||
{
|
|
||||||
pattern: "line",
|
|
||||||
path: tmp.path,
|
|
||||||
},
|
|
||||||
ctx,
|
|
||||||
),
|
|
||||||
)
|
)
|
||||||
expect(result.metadata.matches).toBeGreaterThan(0)
|
expect(result.metadata.matches).toBeGreaterThan(0)
|
||||||
},
|
}),
|
||||||
})
|
),
|
||||||
})
|
)
|
||||||
})
|
|
||||||
|
|
||||||
describe("CRLF regex handling", () => {
|
|
||||||
test("regex correctly splits Unix line endings", () => {
|
|
||||||
const unixOutput = "file1.txt|1|content1\nfile2.txt|2|content2\nfile3.txt|3|content3"
|
|
||||||
const lines = unixOutput.trim().split(/\r?\n/)
|
|
||||||
expect(lines.length).toBe(3)
|
|
||||||
expect(lines[0]).toBe("file1.txt|1|content1")
|
|
||||||
expect(lines[2]).toBe("file3.txt|3|content3")
|
|
||||||
})
|
|
||||||
|
|
||||||
test("regex correctly splits Windows CRLF line endings", () => {
|
|
||||||
const windowsOutput = "file1.txt|1|content1\r\nfile2.txt|2|content2\r\nfile3.txt|3|content3"
|
|
||||||
const lines = windowsOutput.trim().split(/\r?\n/)
|
|
||||||
expect(lines.length).toBe(3)
|
|
||||||
expect(lines[0]).toBe("file1.txt|1|content1")
|
|
||||||
expect(lines[2]).toBe("file3.txt|3|content3")
|
|
||||||
})
|
|
||||||
|
|
||||||
test("regex handles mixed line endings", () => {
|
|
||||||
const mixedOutput = "file1.txt|1|content1\nfile2.txt|2|content2\r\nfile3.txt|3|content3"
|
|
||||||
const lines = mixedOutput.trim().split(/\r?\n/)
|
|
||||||
expect(lines.length).toBe(3)
|
|
||||||
})
|
|
||||||
})
|
})
|
||||||
|
|||||||
Reference in New Issue
Block a user