Nimble and Databricks
This page covers three ways to use live web data inside Databricks. Nimble is the web data platform that collects the data, and every result lands in your Unity Catalog. For each use case you get a short description, the code to install it in your own workspace and a video of it running.
00_install, enter a catalog, a schema name and your Nimble API key, and click Run all. It takes about two minutes.Before you start, check these points.
Nimble runs the searches a shopper runs on Amazon, Walmart, Target, Foot Locker and ASOS. Every product on the results page lands in one Delta table, with its price, its list price when it is on sale, its rating, its review count and whether the slot is sponsored. A daily job adds a new snapshot, so the table builds a price history.
Pricing, category and e-commerce teams use it. They compare its coverage, prices and freshness with their current provider before the next renewal.
The first run returned 1,913 products from 40 searches in about two minutes. The same day, the alert flagged a Nike Pegasus 41 at $87.49 on Foot Locker, 40% below its $145 list price.
| Notebook | What it creates |
|---|---|
01_build_shelf_feed | The table shelf_products, with 8 running-shoe searches on 5 retailers. You edit the searches in the table shelf_queries. |
02_schedule_and_alert | A daily job and a SQL alert. The alert emails you when a watched shoe sells far below list price or a rival brand buys a sponsored slot on a Nike search. |
03_dashboard | The AI/BI dashboard Running shoe shelf, with price, share on sale, discount depth and sponsored slots by brand and retailer. |
04_compare_with_your_feed | A comparison with a sample of your vendor's file, with coverage, price match and freshness per retailer. |
After the install, open the folder 1_data_feed and run the notebooks in order. The whole folder takes about five minutes.
A Nimble Web Search Agent is an agent that researches the live web for a task you describe. It returns rows in a schema you define. Every value carries the page it came from, called a citation, and a confidence grade, so anyone can check a number in one click.
AI teams and embedded analytics teams use it for ad hoc questions. When a question proves useful, the same notebook runs as a scheduled job.
The earnings agent returned 32 rows, 4 quarters for each of the 8 companies, from each company's own release or filing. 27 rows are graded high confidence on every field, and the other 5 show which fields to check. The reviews agent took about 10 minutes to complete all 8 shoes on the watchlist.
| Notebook | What it creates |
|---|---|
01_competitor_earnings | The table competitor_earnings. It holds quarterly revenue, growth, gross margin and operating income for Nike, adidas, Puma, On, Deckers (HOKA), Lululemon, Under Armour and Asics. Each number comes from the company's own earnings release or filing. |
02_ratings_and_reviews | The table product_reviews. It adds the rating, review count, price, praise and complaints to each shoe in a watchlist table you already have. |
03_genie_space | The Genie space Market research (Nimble). Genie answers questions in plain English from the tables, and calls Nimble for live data when the tables can't answer. |
After the install, open the folder 2_agentic_research. Change the company list or the watchlist to your own, then run the notebooks in order.
MCP (Model Context Protocol) is the standard way AI assistants call outside tools. Nimble's MCP server becomes one connection in Unity Catalog. Genie, Databricks One, AI Playground and Agent Bricks agents all use that connection. Cursor and GitHub Copilot can reach it through the Databricks proxy. Unity Catalog grants decide who can use it, and the audit log records every call with the user and the time.
The platform team that runs Databricks, AWS and Cursor sets it up once. It starts from a business need such as use case 1 or 2.
The connection gives every agent in the workspace 27 Nimble tools. In our tests, the Agent Bricks agent answered a question that needed both the earnings table and a live Amazon search in 78 to 132 seconds.
| Notebook or file | What it creates |
|---|---|
01_mcp_connection | The Unity Catalog connection nimble_mcp, tested with a live search. You can also add Nimble from the Databricks Marketplace. |
02_market_analyst_agent | An Agent Bricks agent, Market analyst (Nimble), with its own serving endpoint. It reads the tables through Genie and calls Nimble for live prices, reviews and news. |
03_governance | Queries on system.access.audit that list every Nimble call and key read by user and day, plus who holds access. |
cursor_mcp.json | The settings that add the same Nimble server to Cursor or GitHub Copilot. |
Run use case 2 first, because the agent uses its Genie space. Then open the folder 3_platform_integration and run the notebooks in order.