network routing

This commit is contained in:
James Long
2026-05-17 14:34:20 -04:00
parent 98689fb125
commit 0ed88cfc21
11 changed files with 503 additions and 44 deletions
+1 -1
View File
@@ -28,7 +28,7 @@ export const Flag = {
OPENCODE_FAKE_VCS: process.env["OPENCODE_FAKE_VCS"],
OPENCODE_SERVER_PASSWORD: process.env["OPENCODE_SERVER_PASSWORD"],
OPENCODE_SERVER_USERNAME: process.env["OPENCODE_SERVER_USERNAME"],
OPENCODE_MOCK: truthy("OPENCODE_MOCK"),
OPENCODE_SIMULATION: truthy("OPENCODE_SIMULATION"),
// Experimental
OPENCODE_EXPERIMENTAL_FILEWATCHER: Config.boolean("OPENCODE_EXPERIMENTAL_FILEWATCHER").pipe(
+1
View File
@@ -104,6 +104,7 @@ export const Provider = Schema.Struct({
})
export type Provider = Schema.Schema.Type<typeof Provider>
export const Catalog = Schema.Record(Schema.String, Provider)
export interface Interface {
readonly get: () => Effect.Effect<Record<string, Provider>>
@@ -59,7 +59,7 @@ Implementation shape:
- Seed it from JSON fixtures supplied through the simulation endpoint or runner config.
- Serialize it into replay traces.
- Fail unsupported operations with typed simulation errors instead of silently falling back to host FS.
- Enable with `OPENCODE_MOCK` for initial startup wiring.
- Enable with `OPENCODE_SIMULATION` for initial startup wiring.
- Use a fixed virtual root, not `process.cwd()`, so host paths are denied by default.
- Use the old branch's Bun preload/plugin redirection only for code paths that bypass `AppFileSystem.Service`.
- Let `sandbox-exec` catch any remaining direct `fs`, `Bun.file`, or process-level filesystem access.
@@ -89,7 +89,7 @@ Todos:
- [x] Define mock filesystem data model and fixture JSON format.
- [x] Implement the `AppFileSystem.Service` layer.
- [x] Add typed errors for unsupported operations and host-FS escapes.
- [x] Add activation path from startup through `OPENCODE_MOCK`.
- [x] Add activation path from startup through `OPENCODE_SIMULATION`.
- [x] Add a tiny fixture that includes `opencode.json`, a workspace root, and a few files.
- [ ] Verify read/glob/grep/write/edit use the mock filesystem.
- [ ] Verify sandbox denies host writes when a bypass is introduced.
@@ -145,11 +145,17 @@ Registration model:
- `SimulationNetwork.Service` owns a registry keyed by method + URL matcher.
- Registry entries should include a `source`/`kind` so failures explain why a URL was allowed or denied.
- Rough implementation exists at `packages/opencode/src/testing/simulation/network.ts`.
- Current rough registry supports exact URL, regex URL, or predicate matchers, optional method filters, parsed request bodies, static responses, dynamic response functions, and full handlers.
- `SimulationNetworkRoutes` imports known schemas from the services that own HTTP call sites and registers schema-backed routes for hardcoded/configurable URL families.
- Configurable/client-provided URLs should be registered through route-family helpers, e.g. `account(baseUrl)`, `models(baseUrl)`, `share(baseUrl)`, `skills(baseUrl)`, and `installation(registryUrl)`.
- Some production schemas are too broad for `Schema.toArbitrary()` today, such as provider catalog fields containing arbitrary mutable JSON. For those cases, the first-pass route can use a narrower generated schema whose values still decode under the production schema.
- Supported entry kinds for the first pass:
- `jsonSchema`: generate JSON from an Effect `Schema` via `toArbitrary()`.
- `text`: return deterministic text/html/markdown content for exact URLs.
- `bytes`: return deterministic binary content for exact URLs.
- `status`: return empty/status-only responses.
- `handler`: inspect method, URL, headers, and parsed body to build a custom response.
- `mcp`: handle JSON-RPC/SSE MCP protocol for a configured MCP server URL.
- `loopback`: allow local app/TUI traffic only.
- Prefer explicit registration at configuration/control boundaries over guessing from arbitrary URLs:
@@ -159,15 +165,24 @@ Registration model:
- Provider model responses come from the mock provider script registry, not generic provider SDK HTTP.
- Unknown non-loopback URLs fail with a typed simulation network error.
Layering caveat:
- Several `defaultLayer`s still provide `FetchHttpClient.layer` internally (`Account`, `ModelsDev`, `ToolRegistry`, `ShareNext`, `SkillDiscovery`, `Instruction`, `Installation`, `Ripgrep`, `Workspace`). A top-level `HttpClient.HttpClient` mock does not necessarily affect those self-contained default layers.
- First-pass startup wiring must either use non-default service layers and provide `SimulationNetwork.layer` once, or make these default layers explicitly mock-aware.
- The same caveat already exists for `AppFileSystem.defaultLayer` in some default layers, so the final simulation startup needs an explicit “normal app with narrow mock boundaries” layer assembly rather than blindly using all default layers.
Todos:
- [x] Locate all backend uses of `HttpClient.HttpClient`, raw `fetch`, provider SDK fetches, webfetch/websearch/share/update paths.
- [x] Classify first-pass network call families into schema-generated, text/bytes, MCP protocol, loopback, and denied.
- [ ] Decide where `toArbitrary()` lives or which package exports it.
- [x] Decide where `toArbitrary()` lives or which package exports it.
- [x] Define rough request matcher shape: exact URL, regex URL, or predicate.
- [x] Add method-aware matching and parsed request body support.
- [x] Define rough schema registration shape for generated responses.
- [x] Add schema-backed route helpers for hardcoded and configurable URL families.
- [ ] Define final schema registration shape for generated responses.
- [ ] Define MCP URL registration from `config.mcp.<name>.url` to an MCP protocol handler.
- [ ] Implement seeded response generation with `toArbitrary()`.
- [x] Implement rough seeded response generation with `Schema.toArbitrary()`.
- [x] Add loopback allowlist handling.
- [x] Add typed simulation error for unregistered non-loopback request.
- [ ] Verify sandbox also blocks external network if mock client is bypassed.
+11 -11
View File
@@ -65,24 +65,24 @@ export type ActiveOrg = {
org: Org
}
class RemoteConfig extends Schema.Class<RemoteConfig>("RemoteConfig")({
export class RemoteConfig extends Schema.Class<RemoteConfig>("RemoteConfig")({
config: Schema.Record(Schema.String, Schema.Json),
}) {}
const DurationFromSeconds = Schema.Number.pipe(
export const DurationFromSeconds = Schema.Number.pipe(
Schema.decodeTo(Schema.Duration, {
decode: SchemaGetter.transform((n) => Duration.seconds(n)),
encode: SchemaGetter.transform((d) => Duration.toSeconds(d)),
}),
)
class TokenRefresh extends Schema.Class<TokenRefresh>("TokenRefresh")({
export class TokenRefresh extends Schema.Class<TokenRefresh>("TokenRefresh")({
access_token: AccessToken,
refresh_token: RefreshToken,
expires_in: DurationFromSeconds,
}) {}
class DeviceAuth extends Schema.Class<DeviceAuth>("DeviceAuth")({
export class DeviceAuth extends Schema.Class<DeviceAuth>("DeviceAuth")({
device_code: DeviceCode,
user_code: UserCode,
verification_uri_complete: Schema.String,
@@ -90,14 +90,14 @@ class DeviceAuth extends Schema.Class<DeviceAuth>("DeviceAuth")({
interval: DurationFromSeconds,
}) {}
class DeviceTokenSuccess extends Schema.Class<DeviceTokenSuccess>("DeviceTokenSuccess")({
export class DeviceTokenSuccess extends Schema.Class<DeviceTokenSuccess>("DeviceTokenSuccess")({
access_token: AccessToken,
refresh_token: RefreshToken,
token_type: Schema.Literal("Bearer"),
expires_in: DurationFromSeconds,
}) {}
class DeviceTokenError extends Schema.Class<DeviceTokenError>("DeviceTokenError")({
export class DeviceTokenError extends Schema.Class<DeviceTokenError>("DeviceTokenError")({
error: Schema.String,
error_description: Schema.String,
}) {
@@ -110,22 +110,22 @@ class DeviceTokenError extends Schema.Class<DeviceTokenError>("DeviceTokenError"
}
}
const DeviceToken = Schema.Union([DeviceTokenSuccess, DeviceTokenError])
export const DeviceToken = Schema.Union([DeviceTokenSuccess, DeviceTokenError])
class User extends Schema.Class<User>("User")({
export class User extends Schema.Class<User>("User")({
id: AccountID,
email: Schema.String,
}) {}
class ClientId extends Schema.Class<ClientId>("ClientId")({ client_id: Schema.String }) {}
export class ClientId extends Schema.Class<ClientId>("ClientId")({ client_id: Schema.String }) {}
class DeviceTokenRequest extends Schema.Class<DeviceTokenRequest>("DeviceTokenRequest")({
export class DeviceTokenRequest extends Schema.Class<DeviceTokenRequest>("DeviceTokenRequest")({
grant_type: Schema.String,
device_code: DeviceCode,
client_id: Schema.String,
}) {}
class TokenRefreshRequest extends Schema.Class<TokenRefreshRequest>("TokenRefreshRequest")({
export class TokenRefreshRequest extends Schema.Class<TokenRefreshRequest>("TokenRefreshRequest")({
grant_type: Schema.String,
refresh_token: RefreshToken,
client_id: Schema.String,
+6 -6
View File
@@ -69,16 +69,16 @@ export class UpgradeFailedError extends Schema.TaggedErrorClass<UpgradeFailedErr
}) {}
// Response schemas for external version APIs
const GitHubRelease = Schema.Struct({ tag_name: Schema.String })
const NpmPackage = Schema.Struct({ version: Schema.String })
const BrewFormula = Schema.Struct({ versions: Schema.Struct({ stable: Schema.String }) })
const BrewInfoV2 = Schema.Struct({
export const GitHubRelease = Schema.Struct({ tag_name: Schema.String })
export const NpmPackage = Schema.Struct({ version: Schema.String })
export const BrewFormula = Schema.Struct({ versions: Schema.Struct({ stable: Schema.String }) })
export const BrewInfoV2 = Schema.Struct({
formulae: Schema.Array(Schema.Struct({ versions: Schema.Struct({ stable: Schema.String }) })),
})
const ChocoPackage = Schema.Struct({
export const ChocoPackage = Schema.Struct({
d: Schema.Struct({ results: Schema.Array(Schema.Struct({ Version: Schema.String })) }),
})
const ScoopManifest = NpmPackage
export const ScoopManifest = NpmPackage
export interface Interface {
readonly info: () => Effect.Effect<Info>
@@ -1,4 +1,5 @@
import { Config as EffectConfig, Context, Effect, Layer } from "effect"
import { NodePath } from "@effect/platform-node"
import { HttpApiBuilder, OpenApi } from "effect/unstable/httpapi"
import {
FetchHttpClient,
@@ -12,14 +13,17 @@ import * as Socket from "effect/unstable/socket/Socket"
import { AppFileSystem } from "@opencode-ai/core/filesystem"
import { Flag } from "@opencode-ai/core/flag/flag"
import { Account } from "@/account/account"
import { AccountRepo } from "@/account/repo"
import { Agent } from "@/agent/agent"
import { Auth } from "@/auth"
import { Bus } from "@/bus"
import { CrossSpawnSpawner } from "@opencode-ai/core/cross-spawn-spawner"
import { Config } from "@/config/config"
import { Command } from "@/command"
import * as Observability from "@opencode-ai/core/effect/observability"
import { File } from "@/file"
import { FileWatcher } from "@/file/watcher"
import { Git } from "@/git"
import { Ripgrep } from "@/file/ripgrep"
import { Format } from "@/format"
import { RuntimeFlags } from "@/effect/runtime-flags"
@@ -28,6 +32,8 @@ import { MCP } from "@/mcp"
import { Permission } from "@/permission"
import { Installation } from "@/installation"
import { InstanceLayer } from "@/project/instance-layer"
import { Npm } from "@opencode-ai/core/npm"
import { Env } from "@/env"
import { Plugin } from "@/plugin"
import { Project } from "@/project/project"
import { ProviderAuth } from "@/provider/auth"
@@ -57,6 +63,7 @@ import { Worktree } from "@/worktree"
import { Workspace } from "@/control-plane/workspace"
import { SimulationFileSystem } from "@/testing/simulation/filesystem"
import { SimulationNetwork } from "@/testing/simulation/network"
import { SimulationNetworkRoutes } from "@/testing/simulation/network-routes"
import { CorsConfig, isAllowedCorsOrigin, type CorsOptions } from "@/server/cors"
import { serveUIEffect } from "@/server/shared/ui"
import { ServerAuth } from "@/server/auth"
@@ -182,7 +189,7 @@ type RouteRequirements =
| HttpRouter.Request<"Requires", unknown>
| HttpRouter.Request<"GlobalRequires", never>
export function createRoutes(
function createProductionRoutes(
corsOptions?: CorsOptions,
): Layer.Layer<never, EffectConfig.ConfigError, RouteRequirements> {
return Layer.mergeAll(rootApiRoutes, eventApiRoutes, instanceRoutes, docRoute, uiRoute).pipe(
@@ -233,8 +240,8 @@ export function createRoutes(
Workspace.defaultLayer,
Worktree.appLayer,
Bus.layer,
Flag.OPENCODE_MOCK ? SimulationFileSystem.layer({ root: "/opencode" }) : AppFileSystem.defaultLayer,
Flag.OPENCODE_MOCK ? SimulationNetwork.denyUnknownLayer : FetchHttpClient.layer,
AppFileSystem.defaultLayer,
FetchHttpClient.layer,
HttpServer.layerServices,
]),
Layer.provide(Layer.succeed(CorsConfig)(corsOptions)),
@@ -243,6 +250,88 @@ export function createRoutes(
)
}
export function createSimulatedRoutes(corsOptions?: CorsOptions): ReturnType<typeof createProductionRoutes> {
const simulationBoundary = Layer.mergeAll(
SimulationFileSystem.layer({ root: "/opencode" }),
SimulationNetwork.layer({ entries: SimulationNetworkRoutes.defaults(), allowLoopback: true }),
)
const simulatedServices = Layer.mergeAll(
errorLayer,
compressionLayer,
corsVaryFix,
fenceLayer,
cors(corsOptions),
AccountRepo.layer,
Account.layer,
Agent.layer,
Auth.layer,
Command.layer,
Config.layer,
File.layer,
FileWatcher.layer,
Format.layer,
Git.layer,
LSP.layer,
Installation.layer,
MCP.layer,
ModelsDev.layer,
Npm.layer,
Env.layer,
Permission.layer,
Plugin.layer,
Project.layer,
ProviderAuth.layer,
Provider.layer,
Pty.layer,
PtyTicket.layer,
Question.layer,
Ripgrep.layer,
Session.layer,
SessionCompaction.layer,
SessionPrompt.layer,
SessionRevert.layer,
SessionShare.layer,
SessionRunState.layer,
SessionStatus.layer,
SessionSummary.layer,
ShareNext.layer,
Snapshot.layer,
SyncEvent.layer,
Skill.layer,
Todo.layer,
ToolRegistry.layer,
Vcs.layer,
Workspace.layer,
Worktree.layer,
Bus.layer,
HttpServer.layerServices,
).pipe(
Layer.provideMerge(AccountRepo.layer),
Layer.provideMerge(Git.layer),
Layer.provideMerge(Npm.layer),
Layer.provideMerge(Env.layer),
Layer.provide(CrossSpawnSpawner.layer),
Layer.provide(NodePath.layer),
Layer.provideMerge(simulationBoundary),
)
return Layer.mergeAll(rootApiRoutes, eventApiRoutes, instanceRoutes, docRoute, uiRoute).pipe(
Layer.provide(simulatedServices),
Layer.provideMerge(Layer.succeed(CorsConfig)(corsOptions)),
Layer.provideMerge(InstanceLayer.layer),
Layer.provideMerge(Observability.layer),
) as ReturnType<typeof createProductionRoutes>
}
export function createRoutes(corsOptions?: CorsOptions) {
if (Flag.OPENCODE_SIMULATION) {
return createSimulatedRoutes(corsOptions)
}
return createProductionRoutes(corsOptions)
}
export const routes = createRoutes()
export const webHandler = lazy(() =>
+1 -1
View File
@@ -31,7 +31,7 @@ export type Req = {
baseUrl: string
}
const ShareSchema = Schema.Struct({
export const ShareSchema = Schema.Struct({
id: Schema.String,
url: Schema.String,
secret: Schema.String,
+2 -2
View File
@@ -9,12 +9,12 @@ import * as Log from "@opencode-ai/core/util/log"
const skillConcurrency = 4
const fileConcurrency = 8
class IndexSkill extends Schema.Class<IndexSkill>("IndexSkill")({
export class IndexSkill extends Schema.Class<IndexSkill>("IndexSkill")({
name: Schema.String,
files: Schema.Array(Schema.String),
}) {}
class Index extends Schema.Class<Index>("Index")({
export class Index extends Schema.Class<Index>("Index")({
skills: Schema.Array(IndexSkill),
}) {}
@@ -0,0 +1,101 @@
import { Schema } from "effect"
import { Account } from "@/account/account"
import { Installation } from "@/installation"
import { ShareNext } from "@/share/share-next"
import { Discovery } from "@/skill/discovery"
import { SimulationNetwork, type ResponseEntry } from "./network"
function trim(url: string) {
return url.replace(/\/+$/, "")
}
const GeneratedModel = Schema.Struct({
id: Schema.String,
name: Schema.String,
release_date: Schema.String,
attachment: Schema.Boolean,
reasoning: Schema.Boolean,
temperature: Schema.Boolean,
tool_call: Schema.Boolean,
limit: Schema.Struct({
context: Schema.Finite,
output: Schema.Finite,
}),
})
const GeneratedProviderCatalog = Schema.Record(
Schema.String,
Schema.Struct({
name: Schema.String,
env: Schema.Array(Schema.String),
id: Schema.String,
models: Schema.Record(Schema.String, GeneratedModel),
}),
)
export function account(baseUrl: string): ResponseEntry[] {
const base = trim(baseUrl)
return [
SimulationNetwork.jsonSchema({ method: "POST", url: `${base}/auth/device/code` }, Account.DeviceAuth),
SimulationNetwork.jsonSchema({ method: "POST", url: `${base}/auth/device/token` }, Account.DeviceToken),
SimulationNetwork.jsonSchema({ method: "GET", url: `${base}/api/orgs` }, Schema.Array(Account.Org)),
SimulationNetwork.jsonSchema({ method: "GET", url: `${base}/api/user` }, Account.User),
SimulationNetwork.jsonSchema({ method: "GET", url: `${base}/api/config` }, Account.RemoteConfig),
]
}
export function models(baseUrl = "https://models.dev"): ResponseEntry[] {
return [SimulationNetwork.jsonSchema({ method: "GET", url: `${trim(baseUrl)}/api.json` }, GeneratedProviderCatalog)]
}
export function share(baseUrl = "https://opncd.ai"): ResponseEntry[] {
const base = trim(baseUrl)
return [
SimulationNetwork.jsonSchema({ method: "POST", url: `${base}/api/share` }, ShareNext.ShareSchema),
SimulationNetwork.status({ method: "POST", url: /\/api\/share\/[^/]+\/sync$/ }, 204),
SimulationNetwork.status({ method: "DELETE", url: /\/api\/share\/[^/]+$/ }, 204),
SimulationNetwork.jsonSchema({ method: "POST", url: `${base}/api/shares` }, ShareNext.ShareSchema),
SimulationNetwork.status({ method: "POST", url: /\/api\/shares\/[^/]+\/sync$/ }, 204),
SimulationNetwork.status({ method: "DELETE", url: /\/api\/shares\/[^/]+$/ }, 204),
]
}
export function skills(baseUrl: string): ResponseEntry[] {
const base = trim(baseUrl)
return [
SimulationNetwork.jsonSchema({ method: "GET", url: `${base}/index.json` }, Discovery.Index),
SimulationNetwork.text({ method: "GET", url: /\/SKILL\.md$/ }, "# Generated Skill\n"),
]
}
export function installation(registryUrl = "https://registry.npmjs.org"): ResponseEntry[] {
return [
SimulationNetwork.text({ method: "GET", url: "https://opencode.ai/install" }, "#!/usr/bin/env bash\n"),
SimulationNetwork.jsonSchema(
{ method: "GET", url: "https://formulae.brew.sh/api/formula/opencode.json" },
Installation.BrewFormula,
),
SimulationNetwork.jsonSchema(
{ method: "GET", url: `${trim(registryUrl)}/opencode-ai/latest` },
Installation.NpmPackage,
),
SimulationNetwork.jsonSchema(
{ method: "GET", url: "https://community.chocolatey.org/api/v2/Packages?$filter=Id%20eq%20%27opencode%27%20and%20IsLatestVersion&$select=Version" },
Installation.ChocoPackage,
),
SimulationNetwork.jsonSchema(
{ method: "GET", url: "https://raw.githubusercontent.com/ScoopInstaller/Main/master/bucket/opencode.json" },
Installation.ScoopManifest,
),
SimulationNetwork.jsonSchema(
{ method: "GET", url: "https://api.github.com/repos/anomalyco/opencode/releases/latest" },
Installation.GitHubRelease,
),
]
}
export function defaults() {
return [...models(), ...share(), ...installation()]
}
export * as SimulationNetworkRoutes from "./network-routes"
@@ -1,15 +1,37 @@
import { Context, Effect, Layer, Ref, Schema } from "effect"
import { Context, Effect, Layer, Ref, Schema, Stream } from "effect"
import { FastCheck } from "effect/testing"
import { HttpClient, HttpClientError, HttpClientResponse } from "effect/unstable/http"
type Matcher = string | RegExp | ((request: RequestInfo) => boolean)
type UrlMatcher = string | RegExp | ((request: RequestInfo) => boolean)
export interface Matcher {
readonly method?: string | readonly string[]
readonly url: UrlMatcher
}
export type RequestBody =
| { readonly type: "empty" }
| { readonly type: "text"; readonly text: string; readonly json?: unknown }
| { readonly type: "bytes"; readonly bytes: Uint8Array }
| { readonly type: "form"; readonly form: FormData }
| { readonly type: "unknown"; readonly value: unknown }
export interface RequestInfo {
readonly method: string
readonly url: URL
readonly headers: Readonly<Record<string, string>>
readonly body: RequestBody
}
export type ResponseEntry =
| {
readonly kind: "jsonSchema"
readonly matcher: Matcher
readonly status?: number
readonly headers?: Readonly<Record<string, string>>
readonly schema: Schema.Codec<unknown, unknown, unknown, never>
readonly seed?: number | ((request: RequestInfo) => number)
}
| {
readonly kind: "json"
readonly matcher: Matcher
@@ -31,6 +53,11 @@ export type ResponseEntry =
readonly headers?: Readonly<Record<string, string>>
readonly body: Uint8Array | ((request: RequestInfo) => Uint8Array)
}
| {
readonly kind: "handler"
readonly matcher: Matcher
readonly handle: (request: RequestInfo) => Response | Promise<Response> | Effect.Effect<Response, SimulationNetworkError>
}
| {
readonly kind: "status"
readonly matcher: Matcher
@@ -64,12 +91,23 @@ export interface Interface {
export class Service extends Context.Service<Service, Interface>()("@opencode/SimulationNetwork") {}
function matches(matcher: Matcher, request: RequestInfo) {
function normalizeMatcher(matcher: Matcher | UrlMatcher): Matcher {
if (typeof matcher === "object" && !(matcher instanceof RegExp) && "url" in matcher) return matcher
return { url: matcher }
}
function matchesUrl(matcher: UrlMatcher, request: RequestInfo) {
if (typeof matcher === "string") return request.url.toString() === matcher
if (matcher instanceof RegExp) return matcher.test(request.url.toString())
return matcher(request)
}
function matches(matcher: Matcher, request: RequestInfo) {
const methods = matcher.method === undefined ? [] : Array.isArray(matcher.method) ? matcher.method : [matcher.method]
if (methods.length > 0 && !methods.some((method) => method.toUpperCase() === request.method.toUpperCase())) return false
return matchesUrl(matcher.url, request)
}
function isLoopback(url: URL) {
return url.hostname === "localhost" || url.hostname === "127.0.0.1" || url.hostname === "::1"
}
@@ -78,8 +116,25 @@ function headers(input: Readonly<Record<string, string>> | undefined, contentTyp
return new Headers({ ...(contentType ? { "content-type": contentType } : {}), ...input })
}
function seedFromRequest(request: RequestInfo) {
return [...`${request.method} ${request.url}`].reduce((acc, char) => (acc * 31 + char.charCodeAt(0)) | 0, 1)
}
function generated(schema: Schema.Codec<unknown, unknown, unknown, never>, seed: number) {
const sample = FastCheck.sample(Schema.toArbitrary(schema), { seed, numRuns: 1 })[0]
return Schema.encodeUnknownSync(schema)(sample)
}
function response(entry: ResponseEntry, request: RequestInfo) {
switch (entry.kind) {
case "jsonSchema":
return new Response(
JSON.stringify(generated(entry.schema, typeof entry.seed === "function" ? entry.seed(request) : (entry.seed ?? seedFromRequest(request)))),
{
status: entry.status ?? 200,
headers: headers(entry.headers, "application/json"),
},
)
case "json":
return new Response(JSON.stringify(typeof entry.body === "function" ? entry.body(request) : entry.body), {
status: entry.status ?? 200,
@@ -95,13 +150,81 @@ function response(entry: ResponseEntry, request: RequestInfo) {
status: entry.status ?? 200,
headers: headers(entry.headers, "application/octet-stream"),
})
case "handler":
return entry.handle(request)
case "status":
return new Response(null, { status: entry.status, headers: headers(entry.headers) })
}
}
function toRequestInfo(method: string, url: URL, headers: Readonly<Record<string, string>>): RequestInfo {
return { method, url, headers }
function responseEffect(entry: ResponseEntry, request: RequestInfo): Effect.Effect<Response, SimulationNetworkError> {
const result = response(entry, request)
if (Effect.isEffect(result)) return result
if (result instanceof Promise) return Effect.promise(() => result)
return Effect.succeed(result)
}
function parseJson(text: string) {
try {
return JSON.parse(text)
} catch {
return undefined
}
}
function asBytes(value: unknown) {
if (value instanceof Uint8Array) return value
if (value instanceof ArrayBuffer) return new Uint8Array(value)
if (typeof value === "string") return new TextEncoder().encode(value)
return undefined
}
function requestBody(body: Parameters<typeof HttpClientResponse.fromWeb>[0]["body"]) {
switch (body._tag) {
case "Empty":
return Effect.succeed({ type: "empty" } satisfies RequestBody)
case "Raw": {
const bytes = asBytes(body.body)
if (!bytes) return Effect.succeed({ type: "unknown", value: body.body } satisfies RequestBody)
const text = new TextDecoder().decode(bytes)
return Effect.succeed({ type: "text", text, json: parseJson(text) } satisfies RequestBody)
}
case "Uint8Array": {
const text = new TextDecoder().decode(body.body)
if (body.contentType.includes("json") || body.contentType.startsWith("text/")) {
return Effect.succeed({ type: "text", text, json: parseJson(text) } satisfies RequestBody)
}
return Effect.succeed({ type: "bytes", bytes: body.body } satisfies RequestBody)
}
case "FormData":
return Effect.succeed({ type: "form", form: body.formData } satisfies RequestBody)
case "Stream":
return Stream.runCollect(body.stream).pipe(
Effect.map((chunks) => {
const bytes = new Uint8Array(chunks.reduce((sum, chunk) => sum + chunk.length, 0))
let offset = 0
for (const chunk of chunks) {
bytes.set(chunk, offset)
offset += chunk.length
}
const text = new TextDecoder().decode(bytes)
if (body.contentType.includes("json") || body.contentType.startsWith("text/")) {
return { type: "text", text, json: parseJson(text) } satisfies RequestBody
}
return { type: "bytes", bytes } satisfies RequestBody
}),
Effect.catch(() => Effect.succeed({ type: "unknown", value: body } satisfies RequestBody)),
)
}
}
function toRequestInfo(
method: string,
url: URL,
headers: Readonly<Record<string, string>>,
body: RequestBody,
): RequestInfo {
return { method, url, headers, body }
}
function toHttpClientError(request: Parameters<typeof HttpClientResponse.fromWeb>[0], error: SimulationNetworkError) {
@@ -127,7 +250,7 @@ export function make(options: Options = {}) {
const handle = Effect.fn("SimulationNetwork.handle")(function* (request: RequestInfo) {
const current = yield* Ref.get(state)
const entry = current.entries.find((entry) => matches(entry.matcher, request))
if (entry) return response(entry, request)
if (entry) return yield* responseEffect(entry, request)
if (current.allowLoopback && isLoopback(request.url)) {
return yield* Effect.promise(() => fetch(request.url, { method: request.method, headers: request.headers }))
}
@@ -150,8 +273,9 @@ export const httpClientLayer = Layer.effect(
const network = yield* Service
return HttpClient.make((request, url) =>
Effect.gen(function* () {
const body = yield* requestBody(request.body)
const response = yield* network
.handle(toRequestInfo(request.method, url, request.headers))
.handle(toRequestInfo(request.method, url, request.headers, body))
.pipe(Effect.mapError((error) => toHttpClientError(request, error)))
return HttpClientResponse.fromWeb(request, response)
}),
@@ -167,26 +291,37 @@ export const layer = (options?: Options) => {
export const denyUnknownLayer = layer({ allowLoopback: true })
export const text = (
matcher: Matcher,
matcher: Matcher | UrlMatcher,
body: string | ((request: RequestInfo) => string),
options?: { status?: number; headers?: Record<string, string> },
) =>
({ kind: "text", matcher, body, ...options }) satisfies ResponseEntry
({ kind: "text", matcher: normalizeMatcher(matcher), body, ...options }) satisfies ResponseEntry
export const json = (
matcher: Matcher,
matcher: Matcher | UrlMatcher,
body: unknown | ((request: RequestInfo) => unknown),
options?: { status?: number; headers?: Record<string, string> },
) =>
({ kind: "json", matcher, body, ...options }) satisfies ResponseEntry
({ kind: "json", matcher: normalizeMatcher(matcher), body, ...options }) satisfies ResponseEntry
export const jsonSchema = (
matcher: Matcher | UrlMatcher,
schema: Schema.Codec<unknown, unknown, unknown, never>,
options?: { status?: number; headers?: Record<string, string>; seed?: number | ((request: RequestInfo) => number) },
) => ({ kind: "jsonSchema", matcher: normalizeMatcher(matcher), schema, ...options }) satisfies ResponseEntry
export const bytes = (
matcher: Matcher,
matcher: Matcher | UrlMatcher,
body: Uint8Array | ((request: RequestInfo) => Uint8Array),
options?: { status?: number; headers?: Record<string, string> },
) => ({ kind: "bytes", matcher, body, ...options }) satisfies ResponseEntry
) => ({ kind: "bytes", matcher: normalizeMatcher(matcher), body, ...options }) satisfies ResponseEntry
export const status = (matcher: Matcher, code: number, options?: { headers?: Record<string, string> }) =>
({ kind: "status", matcher, status: code, ...options }) satisfies ResponseEntry
export const handler = (
matcher: Matcher | UrlMatcher,
handle: (request: RequestInfo) => Response | Promise<Response> | Effect.Effect<Response, SimulationNetworkError>,
) => ({ kind: "handler", matcher: normalizeMatcher(matcher), handle }) satisfies ResponseEntry
export const status = (matcher: Matcher | UrlMatcher, code: number, options?: { headers?: Record<string, string> }) =>
({ kind: "status", matcher: normalizeMatcher(matcher), status: code, ...options }) satisfies ResponseEntry
export * as SimulationNetwork from "./network"
@@ -1,7 +1,8 @@
import { describe, expect } from "bun:test"
import { Effect, Exit, Layer } from "effect"
import { Effect, Exit, Layer, Schema } from "effect"
import { HttpClient, HttpClientRequest, HttpClientResponse } from "effect/unstable/http"
import { SimulationNetwork, type RequestInfo } from "../../../src/testing/simulation/network"
import { SimulationNetworkRoutes } from "../../../src/testing/simulation/network-routes"
import { testEffect } from "../../lib/effect"
const it = testEffect(
@@ -20,6 +21,23 @@ const it = testEffect(
SimulationNetwork.bytes(/https:\/\/example\.com\/echo-bytes/, (request: RequestInfo) =>
new TextEncoder().encode(`bytes:${request.url.searchParams.get("value")}`),
),
SimulationNetwork.json(
{ method: "POST", url: "https://example.com/body" },
(request: RequestInfo) => ({
body: request.body.type === "text" ? request.body.json : null,
}),
),
SimulationNetwork.status({ method: "GET", url: "https://example.com/method" }, 204),
SimulationNetwork.handler({ method: "POST", url: "https://example.com/handler" }, (request: RequestInfo) =>
new Response(JSON.stringify({ handled: request.body.type === "text" ? request.body.json : null }), {
headers: { "content-type": "application/json" },
}),
),
SimulationNetwork.jsonSchema(
{ method: "GET", url: "https://example.com/schema" },
Schema.Struct({ name: Schema.String, ok: Schema.Boolean }),
),
...SimulationNetworkRoutes.models("https://models.example"),
],
}),
)
@@ -81,6 +99,63 @@ describe("SimulationNetwork", () => {
}),
)
it.effect("matches methods before serving responses", () =>
Effect.gen(function* () {
const http = yield* HttpClient.HttpClient
const ok = yield* http.execute(HttpClientRequest.get("https://example.com/method"))
const miss = yield* http.execute(HttpClientRequest.post("https://example.com/method")).pipe(Effect.exit)
expect(ok.status).toBe(204)
expect(Exit.isFailure(miss)).toBe(true)
}),
)
it.effect("passes parsed JSON request bodies to dynamic responses", () =>
Effect.gen(function* () {
const http = yield* HttpClient.HttpClient
const request = HttpClientRequest.post("https://example.com/body").pipe(
HttpClientRequest.bodyJsonUnsafe({ query: "hello" }),
)
const response = yield* http.execute(request)
expect(yield* response.json).toEqual({ body: { query: "hello" } })
}),
)
it.effect("supports full response handlers", () =>
Effect.gen(function* () {
const http = yield* HttpClient.HttpClient
const request = HttpClientRequest.post("https://example.com/handler").pipe(
HttpClientRequest.bodyJsonUnsafe({ value: 42 }),
)
const response = yield* http.execute(request)
expect(yield* response.json).toEqual({ handled: { value: 42 } })
}),
)
it.effect("generates JSON from registered schemas", () =>
Effect.gen(function* () {
const http = yield* HttpClient.HttpClient
const response = yield* http.execute(HttpClientRequest.get("https://example.com/schema"))
expect(response.status).toBe(200)
expect(yield* Schema.decodeUnknownEffect(Schema.Struct({ name: Schema.String, ok: Schema.Boolean }))(
yield* response.json,
)).toEqual(expect.objectContaining({ ok: expect.any(Boolean) }))
}),
)
it.effect("registers known schema-backed route families", () =>
Effect.gen(function* () {
const http = yield* HttpClient.HttpClient
const response = yield* http.execute(HttpClientRequest.get("https://models.example/api.json"))
expect(response.status).toBe(200)
expect(typeof (yield* response.json)).toBe("object")
}),
)
it.effect("can register responses after layer startup", () =>
Effect.gen(function* () {
const network = yield* SimulationNetwork.Service
@@ -93,6 +168,18 @@ describe("SimulationNetwork", () => {
}),
)
it.effect("register adds static JSON responses after startup", () =>
Effect.gen(function* () {
const network = yield* SimulationNetwork.Service
const http = yield* HttpClient.HttpClient
yield* network.register(SimulationNetwork.json("https://opencode.ai/static", { ok: true }))
const response = yield* http.execute(HttpClientRequest.get("https://opencode.ai/static"))
expect(yield* response.json).toEqual({ ok: true })
}),
)
it.effect("can register dynamic responses after layer startup", () =>
Effect.gen(function* () {
const network = yield* SimulationNetwork.Service
@@ -111,4 +198,35 @@ describe("SimulationNetwork", () => {
expect(yield* response.json).toEqual({ host: "opencode.ai", header: "ok" })
}),
)
it.effect("register respects method-specific matchers", () =>
Effect.gen(function* () {
const network = yield* SimulationNetwork.Service
const http = yield* HttpClient.HttpClient
yield* network.register(SimulationNetwork.text({ method: "POST", url: "https://opencode.ai/method" }, "posted"))
const wrongMethod = yield* http.execute(HttpClientRequest.get("https://opencode.ai/method")).pipe(Effect.exit)
const response = yield* http.execute(HttpClientRequest.post("https://opencode.ai/method"))
expect(Exit.isFailure(wrongMethod)).toBe(true)
expect(yield* response.text).toBe("posted")
}),
)
it.effect("register adds schema-generated responses after startup", () =>
Effect.gen(function* () {
const network = yield* SimulationNetwork.Service
const http = yield* HttpClient.HttpClient
const ResponseSchema = Schema.Struct({ id: Schema.String, enabled: Schema.Boolean })
yield* network.register(SimulationNetwork.jsonSchema("https://opencode.ai/generated", ResponseSchema))
const response = yield* http.execute(HttpClientRequest.get("https://opencode.ai/generated"))
const json = yield* response.json
const decoded = yield* Schema.decodeUnknownEffect(ResponseSchema)(json)
expect(decoded).toEqual({ id: expect.any(String), enabled: expect.any(Boolean) })
}),
)
})