simple-data-analysis

Fast DuckDB-powered TypeScript library for tabular analysis, geospatial analysis, network graph analysis, vector similarity search, AI, Google Sheets, and data

What is simple-data-analysis?

Fast DuckDB-powered TypeScript library for tabular analysis, geospatial analysis, network graph analysis, vector similarity search, AI, Google Sheets, and data visualization.

Overview of simple-data-analysis

What it is

Simple Data Analysis (SDA) is a TypeScript library for tabular, geospatial, vector, and AI-assisted data analysis. It runs on Node.js and is powered by DuckDB. The library is split into two packages: simple-data-analysis-core (core DuckDB functions) and simple-data-analysis (extends core with AI, Google Sheets, Google Cloud Storage, and charting features). It is maintained by Nael Shiab, a computational journalist and senior data producer for CBC News. The library is open source under the MIT license.

Key features

  • DuckDB-powered: uses DuckDB for fast analytical queries, with lazy-loaded extensions for spatial, fuzzy matching, full-text search, and vector indexes.
  • Chainable API: synchronous transformation methods queue operations; async observer methods (e.g., getData(), log(), writeData()) flush the queue in a single DuckDB statement.
  • Geospatial analysis: supports spatial joins, point creation, and GeoJSON I/O via the DuckDB spatial extension.
  • AI integration: includes row-by-row AI processing, embeddings, vector similarity search, hybrid search, RAG, and natural language queries.
  • External services: integrates with Google Sheets, Google Cloud Storage, and Datawrapper for charts and maps.
  • Lazy dependencies: AI, Google, and charting dependencies load only when used, avoiding startup cost for core operations.

Use cases

  • Tabular data analysis: loading, filtering, joining, summarizing, and exporting CSV/Parquet/JSON data.
  • Geospatial analysis: processing point data, performing spatial joins, and exporting GeoJSON.
  • Data journalism: end-to-end pipelines from data loading to visualization, all in TypeScript.
  • AI-assisted analysis: using embeddings, vector search, and RAG for natural language queries over datasets.
  • Integration with external services: reading/writing Google Sheets, uploading to Google Cloud Storage, and publishing to Datawrapper.

Getting started

Install the library via npm:bashnpm i @nshiab/simple-data-analysisTo start a new data project with essential folders and configurations:bashnpx @nshiab/setup-data-project

License and pricing

The library is open source under the MIT license. No pricing information is stated.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/nshiabsimple-data-analysis.svg)](https://yunodb.com/tools/nshiabsimple-data-analysis?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/nshiabsimple-data-analysis?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/nshiabsimple-data-analysis.svg" alt="Featured on YunoDB" height="20"></a>