> For the complete documentation index, see [llms.txt](https://swarmnexus.gitbook.io/swarmnexus/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://swarmnexus.gitbook.io/swarmnexus/documentation/readme/technical-documentation/quickstart.md).

# System Architecture

The Swarm Nexus production system is designed for resilience, scalability, and maintainability. This document provides a high-level overview of the production topology, data flow, and how the core components interact.

#### Production Topology

Our infrastructure runs on a dedicated host and is orchestrated to ensure a clear separation of concerns between data collection, data presentation, and API access.

* **Host:** Ubuntu 23.04
* **Web Server:** Nginx, which terminates TLS and acts as a reverse proxy.
* **Data Store:** A single SQLite database running in Write-Ahead Logging (WAL) mode for concurrent read/write access.
* **Services:**
  * **Dashboard:** A Flask/Gunicorn application for the user interface.
  * **Agent:** A Node.js (Hono) application providing a REST API.
  * **Collector:** A Node.js (Puppeteer) service responsible for data ingestion.
* **Orchestration:** Services and recurring tasks (like backups and health checks) are managed by `systemd`.

#### System Diagram

```mermaid
flowchart LR
  subgraph "Host (Ubuntu 23.04)"
    Nginx[Nginx: TLS + Reverse Proxy]
    Dashboard[Dashboard Service:3010]
    Agent[Agent REST API:3011]
    Collector[Collector Service]
    DB[(SQLite Database<br>/opt/swarmnexus/data/swarm.db)]
    Heartbeat[(collector_heartbeat.json)]

    Nginx --> Dashboard
    Nginx --> Agent
    Dashboard --> DB
    Agent --> DB
    Collector --> DB
    Collector --> Heartbeat
  end
  Internet[Users & External Services] --> Nginx
```

#### End-to-End Data Flow

1. **Collection:** The **Collector** service actively scrapes Twitter (X) for mentions of `@SwarmNexus`. It fetches the full context of each mention, parses it for metadata (like `$TICKERs`), and upserts the structured data into the SQLite database. After each successful run, it writes a "heartbeat" file to confirm it is operational.
2. **Storage:** All data resides in a central SQLite database in WAL mode. This allows the **Collector** to write new data without blocking the read-only access required by the other services.
3. **Presentation & Access:**
   * The **Dashboard** (Flask app) reads from the database to provide the user interface, leaderboards, and content feeds.
   * The **Agent** (Node.js API) reads from the database to expose structured data via a public REST API for developers and third-party integrations.
4. **Proxy & Security:** All services listen on the local loopback address (`127.0.0.1`). **Nginx** is the only component exposed to the public internet. It handles all incoming traffic, terminates SSL/TLS, and reverse-proxies requests to the appropriate backend service based on the URL path (`/` for the Dashboard, `/agent/` for the API).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://swarmnexus.gitbook.io/swarmnexus/documentation/readme/technical-documentation/quickstart.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.