Rhys Sullivantrim sizes71e33a98 months agoHistory
132 lines · 7.7 KB · Text
RawDownload
---title: "Import Fidelity"description: "What executor preserves, normalizes, and currently loses when importing external formats."--- Executor should normalize syntax, not meaning. That means the import pipeline can simplify surface differences between OpenAPI, GraphQL, Google Discovery, and MCP, but it should not silently erase semantics that change execution, trust, or user understanding. The practical shape is: - keep the raw source document or manifest- keep adapter-native metadata that does not fit the common model cleanly- promote execution-relevant semantics into first-class IR ## The fidelity buckets This audit uses four buckets: - `Raw source`: the original imported document or manifest- `Adapter/native`: typed provider data or native blobs attached during import- `IR`: first-class catalog symbols and capability metadata- `UI today`: what current inspection surfaces directly Current inspection is still narrow. It mostly shows projected schemas plus `tool.executable.native[0]`, not all preserved native blobs or capability-native metadata. ## OpenAPI | Source detail | Raw source | Adapter/native | IR | UI today | Notes || --- | --- | --- | --- | --- | --- || Operation auth requirements and security schemes | Yes | No | No | No | Operation `security` is not promoted; imported capabilities currently use `auth: none`. || Exact response variants, status codes, and headers | Yes | Partial | No | Partial | Import chooses one preferred response, then projects one synthetic `2XX` response set. || Parameter serialization (`style`, `explode`, `allowReserved`, content-based params) | Yes | Mostly no | No | No | The IR can model these, but the OpenAPI importer currently only takes parameter `schema`. || Multiple request and response media types | Yes | Partial | Partial | Partial | Import picks a preferred content entry for typing; response docs keep content types only for the chosen response. || `servers` defaults | Yes | No | No | No | The IR can represent scope server defaults, but OpenAPI import does not populate them. || Rich schema semantics beyond the importer subset | Yes | Partial | Partial | Partial | Unsupported JSON Schema shapes fall back to `unknown` plus native preservation. | ## GraphQL | Source detail | Raw source | Adapter/native | IR | UI today | Notes || --- | --- | --- | --- | --- | --- || Subscription root fields as tools | Yes | Partial | No | No | The manifest records `subscriptionTypeName`, but tool generation only materializes query and mutation fields. || Full field result shapes for generated field tools | Yes | Partial | Partial | Partial | Field-tool outputs are heuristic projections, not full result types. || Union and deep nested selections | Yes | Partial | Partial | Partial | Unions degrade to `__typename`; nested selection stops after depth 2 in generated field tools. || Arbitrary caller-selected field projections | Yes | Partial | No | No | Generated field tools are compiled with fixed selections; the raw `request` tool is the escape hatch. || Deprecation reasons | Yes | Partial | Partial | No | GraphQL schemas emit `deprecated: true` plus `x-deprecationReason`, but the generic schema importer only keeps `deprecated` as first-class shape metadata. | GraphQL is a special case: the main loss is not the raw source being discarded, but the generated field-tool projection being intentionally narrower than the schema. That tradeoff is reasonable for discoverability, but only if the system continues to preserve and expose the raw request path. ## Google Discovery | Source detail | Raw source | Adapter/native | IR | UI today | Notes || --- | --- | --- | --- | --- | --- || OAuth scope descriptions | Yes | Partial | No | No | Document-level scope descriptions are extracted, but the IR security scheme rewrites scopes to `{ scope: scope }`. || Rich media upload metadata | Yes | Partial | Partial | Partial | Upload support is collapsed to booleans such as `supportsMediaUpload` and `supportsMediaDownload`. || Service-level metadata (`title`, `description`, `batchPath`, `documentationLink`) | Yes | Partial | No | No | The extracted manifest has these fields, but per-operation provider data does not carry them into the catalog. || Parameter descriptions, repeated-ness, enums, defaults | Yes | Yes | Mostly yes | Mostly yes | This area is comparatively good. || Request and response schemas | Yes | Yes | Yes | Yes | Core schema structure is preserved reasonably well. | Google Discovery is less lossy than OpenAPI on request and response typing, but it still loses useful service metadata and the human meaning attached to OAuth scopes. ## MCP | Source detail | Raw source | Adapter/native | IR | UI today | Notes || --- | --- | --- | --- | --- | --- || Non-tool MCP resources such as prompts or resources | No | No | No | No | The adapter only discovers `listTools()`. Anything outside the tool list never enters import. || Extra tool metadata beyond `name`, `description`, `inputSchema` or `parameters`, and `outputSchema` | Partial | Partial | No | No | The manifest schema keeps only a narrow subset of listed tool fields. || Tool-level safety or effect hints, if present | Partial | No | No | No | Imported MCP operations are currently hardcoded to `effect: action`. || Tool input and output schemas | Yes | Yes | Yes | Yes | This is the strongest part of the current MCP import. || Server-level metadata outside the tool list | No | No | No | No | The stored MCP "document" is a synthesized tool manifest, not a richer upstream server description. | MCP is the most structurally lossy adapter today because the discovery step itself only imports tools and already narrows their metadata aggressively. ## Cross-cutting UI caveat Even when data is preserved natively, the current inspection UI often does not show it directly. Today inspection primarily exposes: - projected call and result schemas- capability summary text- the first executable native blob That means "preserved in native data" and "visible to a user" are still different states. ## What should be first-class in IR These semantics are worth promoting into the common model because they affect execution or trust: - auth and security requirements described by the source format- response and status variants- HTTP parameter serialization- upload, download, and streaming traits- GraphQL selection and subscription semantics- source-provided safety or idempotency hints By contrast, many provider-specific labels, service metadata fields, and raw source extras are fine to keep in adapter-native metadata as long as they remain inspectable. ## Recommended priority order ### P0 - OpenAPI auth and security requirements- OpenAPI response variants and status distinctions- Google Discovery scope descriptions- MCP discovery breadth: decide whether executor intends to import only tools or broader MCP surface area ### P1 - OpenAPI parameter serialization- GraphQL subscription modeling- GraphQL field-tool projection metadata so the fixed-selection behavior is explicit- Google Discovery service metadata surfacing- MCP tool metadata preservation beyond the current narrow manifest ### P2 - Better UI fallback to native/provider metadata- Richer diagnostics for intentionally lossy normalization- A fixture-backed audit matrix test per adapter ## The IR boundary The right boundary is: - `raw source` remains the source of truth- `adapter-native metadata` keeps source-specific details close to their format- `IR` stays unified, but it must be lossless for downstream semantics If executor keeps that line clear, a standardized representation is a strength. If executor lets semantically important details live only in hidden native blobs, the standardized representation becomes misleading.