Case Study

Synthetic Web Data Pipeline

A reusable analytics engineering project that generates realistic web analytics data for BI, SQL practice, dbt transformations, dashboarding, and data pipeline demos.

Overview

A realistic analytics sandbox for web data

The project simulates page-level activity for an imaginary website, including sessions, pageviews, signups, newsletter signups, user/device/source behavior, time-based patterns, seasonality, randomness, and configurable behavior through YAML.

Generated Activity

Synthetic sessions, pageviews, account signups, newsletter signups, and page-level behavior for an imaginary website.

Configurable Behavior

Traffic volume, pages, sources, devices, seasonality, growth, randomness, and conversion rules are controlled through project configuration.

Analytics Sandbox

The goal is realistic practice data for dashboards, SQL analysis, dbt transformations, and reporting demos, not fake business claims.

Data generation

How the synthetic data is generated

The generator creates believable analytics records without visiting real websites, scraping, or simulating fake users in a browser.

01

Configuration defines traffic behavior, page structure, event probabilities, seasonality, growth, and randomness.

02

A scheduled Python job generates synthetic daily web analytics records.

03

Generated data is loaded into Postgres at a page-level grain.

04

dbt models transform raw records into reporting-ready analytics tables.

05

The portfolio reads aggregated data server-side and renders a compact live dashboard.

Live dashboard

Generated data snapshot

This dashboard queries the project database server-side and renders only aggregated metrics in the browser.

Web analytics over time

Select a report period and metric to inspect the generated daily data.

Sessions

17,018,002

Pageviews

46,243,601

Signups

585,543

Newsletter signups

415,681

Bounce rate

31.7%

Avg. time

2m 25s

2026-01-01 to 2026-09-12

Total Pageviews: 46,243,601

Daily average Pageviews: 181,347

Last Generated Data

Last generated date: 2026-09-12

Timestamp: Sep 12, 2026, 12:00 AM

Rows generated: 2,250

Status: Success

Test results: Success

Technical stack

Built for analytics engineering practice

The project combines generation, storage, transformation, scheduling, and reporting patterns that mirror real analytics workflows.

View source code
PythonPostgresdbt CoreYAML configurationRailway cron/job schedulingSQLAnalytics engineeringDashboarding / BI