Topic · AI
Retrieval and data
31 developments in the last 30 days.
Retrieval and data AI
10 developments this week
-12 vs last week
Stage: Coverage
Opportunities
- Labs bundle web search for agents, and standalone search APIs lose pricing powerModel labs now sell search and fetch alongside tokens. Agent builders pay less, and independent search-API vendors get squeezed.First flagged 2026-10-18
- Search Shim: a drop-in search-and-fetch tool over DeepSeek's $0.75 API, with fallbackAn open-source MCP server and hosted router that defaults agent web search to DeepSeek's cheap API. Indie agent builders pay.First flagged 2026-10-18
- Gateway Scorecard: independent MCP gateway testing for bank procurementRuns capability and security tests on MCP gateways against a bank's own tools. Bank AI platform teams pay.First flagged 2026-10-19
- Tool Index: one local MCP server that gives an agent hundreds of tools through threeAn open-source gateway with three tools (search, describe, call) over all your MCP servers. Power users and small teams pay for synced configs.First flagged 2026-10-19
- MCP gateways become the agent control point, and now there is a benchmark to compare themMiddleware between agents and tools that picks the right tool, enforces policy and logs every call. Enterprise security teams pay.First flagged 2026-10-07
- Scam Second Opinion: on-device check of suspicious chats and screenshotsBrowser extension and phone app that judge suspicious messages locally with small models. Worried families pay.First flagged 2026-10-15
- Dead-startup data becomes a liquidation asset for AI labsBrokering, cleaning and valuing code and data from wound-down startups, paid for by AI labs and by bankruptcy trustees.First flagged 2026-10-13
- Wind-Down Data Desk: clean, license and sell a dead fintech's data to AI labsTurns a closing startup's code and data into a compliant training-data sale. Wind-down firms and trustees pay.First flagged 2026-10-13
- Wind-Down Data Room: package a dead startup's code for AI-lab buyersSelf-serve tool that scrubs secrets and PII, inventories a closing startup's code and data, and drafts a sale listing. Founders pay.First flagged 2026-10-13
- Swarm Check: does your multi-agent setup beat one agent on your own tasks?Harness that replays your tasks through solo and team agent configs, reporting pass rate and spend; agent builders pay.First flagged 2026-10-12
- Verification becomes the measurable bottleneck, from math proofs to office workChecking, refereeing and review-load measurement for AI output, paid for by research funders, journals and enterprise CFOs.First flagged 2026-10-11
- Compact AML Validation: weighted test sets and rerun checks for LLM transaction monitoringBuilds small, statistically valid test sets and run-to-run flip rates for LLM AML triage; bank model-risk teams pay.First flagged 2026-10-11
- Clip Finder: local footage search for creators using EmbeddingGemma 2Desktop app that indexes a creator's raw footage locally and finds moments by description. Editors pay once.First flagged 2026-10-11
- Feature Forge: turning unstructured SMB data into validated credit featuresUses an LLM offline to write deterministic feature code from long text such as shipment notes, broker emails and statements, so credit models can use those signals without calling an LLM at decision time.First flagged 2026-10-09
- Invoice Sense: does this invoice fit this business, checked on every paymentFlags invoices whose goods, prices or routes don't fit the payer's business. FCA-registered EMIs and payment fintechs pay.First flagged 2026-10-10
- Guardrail Fuzzer for typed decision modelsAn open-source red-team harness that tests whether your allow/deny decision-model guardrail can be pushed into failing open with one-word option and definition attacks.First flagged 2026-10-09
- AI pipeline leakage audit for regulated dataScans a fintech's AI stack for leaks that teams treat as safe: router and gateway logs that reveal sensitive topics, and multi-vector document indexes that can be turned back into the original pages.First flagged 2026-10-08
- Double-pledge detection for invoice finance using on-prem multimodal embeddingsCatches the same invoice, or a lightly altered copy, being financed twice across a lender's portfolio, without sending documents to a third-party cloud.First flagged 2026-10-07
- Fair Shake: matched-pair fairness testing for AI banking assistantsAutomated test personas that probe a bank's AI assistant or adviser for unequal pricing, offers or advice by wealth, age or location, then produce an evidence report regulators will accept.First flagged 2026-10-08
- MAS-ready AI pre-implementation review packA review workbench that turns a bank's planned AI project into the evidence package an independent reviewer needs: risk sizing, test results, agent action logs and sign-offs.First flagged 2026-10-08
Developments
Last 30 days- DeepSeek search and fetch – $0.75 per 1000 successful requestsDeepSeek launched a search-and-fetch API at $0.75 per 1,000 successful requests, which cuts the cost of grounding agents in web results. Hacker News · 2026-10-19
- Tool Recommendation for Multi-Step Agents: Finding Every Tool a Task NeedsA tool-recommendation system now in production lets agents reach about 8,200 tools, far more than fit in any model's context window, which is a practical answer to MCP tool sprawl. arXiv · 2026-10-19
- DeepSeek releases AI model update, opens search APIs to all usersDeepSeek updated its model and opened its search APIs to all users. Investing.com · 2026-10-18
- Private Retrieval Augmented Generation via Random ProjectionApplying the same random projection to stored and query embeddings keeps a RAG system's data private while retrieval still works. arXiv · 2026-10-18
- Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation PipelineOpen 2.8T-token Indic pretraining dataset and generation pipeline released. Hugging Face Blog · 2026-10-17
- Untangling 500-page chemistry patents with Gemini 4Isomorphic Labs uses Gemini 4 to pull structured data out of 500-page chemistry patents. Google DeepMind · 2026-10-16
- Pg_vectorsearch: An index that is faster than pgvector's IVFFlat and HNSWpg_vectorsearch is a new Postgres vector index that its authors say beats pgvector's IVFFlat and HNSW indexes on speed, which could make retrieval inside Postgres cheaper and faster. Hacker News · 2026-10-15
- Bloomberg is making its ASKB conversational AI interface available to all Terminal users, letting them integrate their own content with existing financial data (Steven Rosenbush/Wall Street Journal)Bloomberg opened its ASKB AI assistant to all Terminal users and lets them add their own content to its financial data. Techmeme · 2026-10-15
- Web search for a 1.7B model running fully on a phone, no search API and no backend (open source)Private Mind, an open-source app, adds web search with citations to a 1.7B model running fully on a phone, with no search API or backend. r/LocalLLaMA · 2026-10-15
- Tencent open-sourced Agent Memory and it runs 100% locally on SQLiteTencent open-sourced an agent memory system that runs fully locally on SQLite. r/LocalLLaMA · 2026-10-14
- AI labs are buying data from dead startups to train their modelsAI labs are buying code and data from shut-down startups for training, an emerging data market. MIT Technology Review · 2026-10-14
- Same Query, Different Answer: Run-to-Run Variance in Semantic Query ProcessingLLM semantic query engines give materially different answers on repeated runs, even at temperature 0. arXiv · 2026-10-13
- VectorDash: A full Windows search replacement incorporating Embeddinggemma-2Several Windows semantic search tools built on EmbeddingGemma 2 (VectorDash, FileDig, LSearch) appeared at once, pointing to a crowded niche. r/LocalLLaMA · 2026-10-12
- Open-source Mac app that runs EmbeddingGemma 2 locally to search your files by what’s in themDigUp, an open-source Mac app, uses EmbeddingGemma 2 to search files locally across text, images, audio and video. r/LocalLLaMA · 2026-10-11
- Long Text to Predictive Features: LLM-Guided Blockwise Feature Engineering via Executable Program SearchAn LLM writes executable feature-extraction programs that turn long text into features for risk-control models, so no LLM call is needed on each live input. arXiv · 2026-10-11
- From Public Posts to AI-Search Citations: Measuring the Fragility of AI SearchPublic posts can be used to manipulate which sources AI search engines cite, which points to a need for citation-security and AI-search-optimization tools. arXiv · 2026-10-11
- Internalizer: Portable Context-to-Parameter Mapping for Very Large Language ModelsThe Internalizer is a hypernetwork that turns documents into LoRA adapters for the frozen 284B-parameter DeepSeek v4, well beyond the 14B scale shown in earlier work. arXiv · 2026-10-11
- tencent/Youtu-Parsing-Omni · Hugging FaceTencent released a compact 5B model that parses documents, images, charts, audio and video into one structured JSON output. r/LocalLLaMA · 2026-10-10
- lightonai/LightOnOCR-3-4B · Hugging FaceLightOn released LightOnOCR-3, a 4B open OCR model. r/LocalLLaMA · 2026-10-10
- jevman: AI decision models play Pac-ManA new category of low-latency 'decision models' (OpenAI's decisions endpoint, Cloudflare Clef, Liquid d1) is getting compared by developers on real-time tasks like playing Pac-Man. r/LocalLLaMA · 2026-10-09 NET
- Global PC shipments dropped 20.1% YoY in Q3 to 62.7M units due to higher prices and supply issues; Lenovo dropped 22.6% YoY, HP 30.9%, Dell 25%, and Apple 11.3% (IDC)Global PC shipments fell 20% year over year in Q3 as higher prices driven by memory costs and supply problems hit demand. Techmeme · 2026-10-09 HPQ DELL AAPL
- Sources: Apple told some suppliers to cut iPhone 18 Pro and Pro Max component production by 15%-20% as higher memory costs pushed up prices and hurt demand (Nikkei Asia)Apple has reportedly told suppliers to cut iPhone 18 Pro component orders by 15–20% as higher memory costs hurt demand. Techmeme · 2026-10-09 AAPL
- Multimodal open d1 decision models for the edgeHugging Face blog post on open multimodal d1 decision models built for edge devices. Hugging Face Blog · 2026-10-09
- EmbeddingGemma 2: an open, lightweight multimodal embedding modelGoogle DeepMind released EmbeddingGemma 2, an open, lightweight multimodal embedding model, cheap to self-host for retrieval. Google DeepMind · 2026-10-08 GOOGL
- OpenAI launches Decisions API that reduces complex evaluations to yes, no, or pick oneOpenAI's Decisions API returns yes/no, pick-one or rating answers about 10x faster than the Responses API for $0.10 per million input tokens, so cheap classification for fraud and compliance becomes viable. The Decoder · 2026-10-08
- d1-3B and d1-omni from LiquidAIDetails on d1-omni-600M: it takes text, JSON, images or voice plus a set of named questions and reads typed answers straight from the model, enabling cheap on-device decisions. r/LocalLLaMA · 2026-10-08
- Inverting Multi-Vector Visual Document IndicesResearchers show multi-vector visual document indices can be inverted to recover page content, a privacy risk for hosted vector databases. HF Daily Papers · 2026-10-08
- Does Document Structure Help Dense Retrieval? A Placebo-Controlled Ablation of Four Mechanisms Across Two CorporaA controlled ablation tests whether document-structure tricks like contextual chunks really help dense retrieval, which matters for anyone building RAG. arXiv · 2026-10-08
- llm-openai-decisions 0.1a0Simon Willison released an llm plugin for OpenAI's new Decisions API, an early sign of developers picking it up. Simon Willison · 2026-10-08
- Efficient Provably Private Classification with a Tabular Foundation ModelFast, provably private classification with a tabular foundation model could suit sensitive finance data. arXiv · 2026-10-08
- EmbeddingGemma 2Simon Willison notes EmbeddingGemma 2 is Apache 2.0 and argues closed, hosted-only embedding models make little sense. Simon Willison · 2026-10-08