Datamimic – don’t let your coding agent invent its own test world

Datamimic – don't let your coding agent invent its own test world. Please reload this page. rapiddweller / datamimic Public Notifications You must be signed in to change notification settings Fork 3 Star 44 developmentBranchesTagsGo to fileCodeOpen more actions menuLatest commit History181 Commits181 CommitsFolders and filesNameNameLast commit messageLast commit date. github/workflows. github/workflows benchmarks/dsl-authoringbenchmarks/dsl-authoring datamimic_cedatamimic_ce docsdocs scriptscript tests_cetests_ce. env. example. env. example. gitignore. gitignore AGENTS. mdAGENTS. md CONTRIBUTING. mdCONTRIBUTING. md LICENSELICENSE MANIFEST. inMANIFEST. in MakefileMakefile README. mdREADME. md RELEASE. mdRELEASE. md llms. txtllms. txt local. env. propertieslocal. env. properties pyproject. tomlpyproject. toml sonar-project. propertiessonar-project. properties View all filesRepository files navigationDATAMIMIC — Governed Test Data for Regulated Enterprises This repository contains the DATAMIMIC Community Edition (CE).

What happened

The Enterprise Platform adds governed workflows, PII scanning, role-based access, audit logging, scheduling, multi-system execution, and the full operational layer that regulated enterprises require. 👉 Enterprise Platform: datamimic. io | 📘 Docs: docs. datamimic. io | 📅 Book a strategy call: datamimic. io/contact 🤖 AI agent? Start at AGENTS. md and use the project CLI: preserve new intent as model. dm. json, submit an early best attempt via datamimic scaffold. . . –format json, repair from the structured issues, declare an expectation per stated requirement, and stop on verified=true.

Available in CE (this repo): Generate fully synthetic, deterministic datasets — model-driven, no source data required Pseudonymize staging/QA exports — deterministic (seeded) or privacy-maximized (non-seeded) field transformation; PII fields identified and modeled manually in the XML pipeline Execute single-system pipelines against PostgreSQL · MySQL · Oracle · MS SQL · SQLite · MongoDB · CSV · JSON · XML · XLSX · DbUnit · fixed-width (. fcw) Model behavior — weighted state machines, composite multi-field references, control flow (, ), and a scriptable memstore for staged aggregation Emit provenance — append-only execution logs and per-output content hash for audit re-execution Guide agents — machine-readable capabilities, progressive reference queries, and one canonical CLI scaffold transaction; an optional MCP adapter exposes the same authoring service The Enterprise Platform adds: PII scanner — probability-scored field detection with configurable thresholds via DataWorkbench Multi-system execution — Oracle / MongoDB / Kafka in coordinated workflows with referential integrity Industry message templates — EDIFACT / SWIFT MT / HL7 v2. x / HL7 FHIR generated as deterministic test/training artefacts Governance layer — role-based dashboards, audit trails, approval flows, reusable enterprise templates, scheduler Performance core — Rust fastpath, ML/auto-regressive engine for complex distributions, keyset and manifest building, optimised distributed execution On-premise / air-gapped deployment — podman-compose or Helm, with consulting-led rollout Deployed in regulated EU banking environments for deterministic test data across Oracle, MongoDB, and Kafka pipelines.

Prompts to paste into your agent Author and verify a new model Create the dataset I describe with DATAMIMIC. Enterprise Platform (EE) Role What DATAMIMIC solves QA / Test Manager Eliminate manual test data requests. This is made possible by the new WhatsApp Business Tools MCP, an MCP (Model Context Protocol) server that directly connects an AI coding agent like Claude, Cursor, Codex, or ChatGPT to the WhatsApp Business Platform. Trust me, I’ve tested every one of our top picks in the real world—think busy New York City streets, loud cafes, crowded gyms, and rush-hour subway platforms—and sometimes, the noise cancellation is less effective at doing its one job.

The wider picture

For testing noise cancellation, I use headphones in the real world. DATAMIMIC CE is the open-source deterministic data engine at the core of the DATAMIMIC Enterprise Platform. It is usable standalone for synthetic data generation and PII-aware pseudonymization in any local, CI, or agent-driven workflow. The Enterprise Platform adds the governed workflows, scanners, dashboards, and execution layer that regulated enterprises require for production-scale test-data operations. Reference customers available under NDA — see also datamimic. io case studies.

AI agents: author, verify, and run data models The CLI is the baseline agent contract. Install CE with pip install datamimic-ce; inside this checkout, use. venv/bin/datamimic so a stale global installation cannot change the available schema or commands. Need CLI tool Contract Discover the live structural surface datamimic capabilities Compact machine-readable JSON index by default; –full for the complete manifest, –section for one section.

What has been reported

Learn the Intent Model progressively datamimic reference authoring, then datamimic reference authoring –category –kind Start with the query catalogue, then load only the typed fragment needed. Author a new model Preserve model. dm. json; run datamimic scaffold model. dm. json –format json One compile/lint/bounded-run/acceptance transaction per changed attempt. Work with existing raw XML datamimic lint model. xml –format json, then datamimic dry-run model. xml –format json Fix diagnostics, inspect bounded samples for intent, then use datamimic run model. xml only when real execution is requested.

Find a DSL detail datamimic reference overview, then a narrow reference topic/name Query the live model and rule registries instead of guessing elements, generators, scope, distributions, or rules. capabilities, authoring-reference projections, and the commands shown with –format json return machine-readable JSON. Optional MCP adapter When the calling environment already exposes DATAMIMIC MCP tools, they map to the same canonical contracts and implementations: reference → datamimic_reference, scaffold → datamimic_scaffold, lint → datamimic_check, and dry-run → datamimic_run.

Install the adapter with pip install "datamimic-ce[mcp]"; registration details belong in the MCP quickstart, not in the authoring workflow. In a repository checkout use `. venv/bin/datamimic`; otherwise use the current `datamimic` CLI. Start from the minimal valid document shape in AGENTS. md ("Authoring a new model").

What happens next

Submit EARLY: run `datamimic scaffold model. dm. json –format json` with your best attempt after at most one discovery call. Follow AGENTS. md's "Authoring a new model" and its structural recipes: orders nest inside the customer product's "children" array; the FK field is {"kind": "script", "script": "parent. id"} with a foreign_key role — a randomly generated FK passes schema validation but fails per-parent-count acceptance. Raw XML remains supported for existing descriptors (lint → dry-run → run; see AGENTS. md).

For new models it is a last resort: only when a scaffold issue explicitly classifies the requirement as unsupported_intent should an agent hand-author XML, preserving that evidence. 32) ✅ manual model ✅ automated via DataWorkbench Pseudonymization — non-seeded (privacy-maximized) ✅ manual model ✅ automated via DataWorkbench Python API + XML pipelines ✅ ✅ Domain models: Finance, Healthcare, Demographics ✅ ✅ Time-series generation (, ISO 8601, prefix-stable) ✅ ✅ MCP server for AI agent integration ✅ ✅ CLI + local execution ✅ ✅ Scale millions of records via Python multiprocessing (and optional Ray) designed for billion-record workloads — Rust fastpath, optimised multi-process execution, and keyset/manifest building on top of the shared Ray distribution layer PII scanner ❌ ✅ probability-scored field detection, configurable threshold, DataWorkbench integration Runtime configuration profiles ❌ ✅ Performance · Balanced · Flexibility Memory management standard optimised for high-volume batch and streaming Logging granularity flat execution log configurable: minimal · standard · deep nested tracing Nested structure evaluation basic deep nested generation with extended condition + ruleset evaluation Importer / exporter logging ❌ per-stage logging for importers and exporters Error handling standard exceptions structured error catalog with recovery strategies Rust fastpath ❌ performance-critical paths in Rust Keyset and manifest building ❌ reads live DB schemas to build coordinated multi-table generation plans ML / auto-regressive engine ❌ combine statistical models with conditions, rulesets, validators for complex distributions Platform capabilities (EE only) Capability EE Multi-user collaboration ✅ Role-based access control (RBAC) ✅ Audit logs + provenance dashboards ✅ PII scanner — probability scoring, threshold-based field flagging ✅ DataWorkbench — visual field mapping and pseudonymization model builder ✅ Reusable enterprise template library ✅ Scheduled execution + task runner ✅ CI/CD pipeline integration (Tosca, Jenkins, GitLab) ✅ Multi-system execution: Oracle, MongoDB, Kafka ✅ Template engine: schema-aware editors for EDIFACT, SWIFT MT, HL7 v2. x, and HL7 FHIR — customer-uploadable specs, further industry formats built per engagement on the same framework ✅ Audit-evidence artefacts for GDPR Art.

5 (test data) reviews, and — for US Covered Entities / Business Associates — HIPAA §164. EE template engine The EE template engine generates industry-standard financial messages from DATAMIMIC models.

The report has been compiled by The Daily Waves using information reported across github.com, techcrunch.com, theguardian.com, wired.com. Details are presented according to the information available at the time of publication and may change as authorities, organisers or other relevant parties provide updates.