New User Onboarding Architecture
Complete auto-discovery channel architecture, process flows, database updates, and automation for every new CloudCFO subscriber.
Discovery Channels Recap
Summary of all user receipt capture methods, their user experience, and development effort.
| Method | User Experience | Dev. Effort | Type | n8n Nodes | Per-User Storage |
|---|---|---|---|---|---|
| 📱 Mobile Notification Access | Fully automatic (0 taps) — 3-layer hybrid | High | Auto | 1 webhook | Auth token |
| ↳ Layer 1: Notification text | Vendor + amount (always captured) | 100% | |||
| ↳ Layer 2: MediaStore scan | Full image from gallery (automatic) | ~60% | |||
| ↳ Layer 3: Smart prompt | User shares receipt (1 tap fallback) | +25% | |||
| 📧 Gmail OAuth | Fully automatic | Medium | Auto | 1 webhook | OAuth token |
| 📬 Outlook OAuth | Fully automatic | Medium | Auto | 1 webhook | OAuth token |
| 📁 Google Drive | Fully automatic (entire Drive watched) | Medium | Auto | 1 webhook | OAuth token |
| 📦 Dropbox | Fully automatic | Medium | Auto | 1 webhook | OAuth token |
| ☁️ OneDrive | Fully automatic | Medium | Auto | 1 webhook | OAuth token |
| 💬 Slack | Fully automatic | Medium | Auto | 1 webhook | OAuth token |
| 🟦 Microsoft Teams | Fully automatic | Medium | Auto | 1 webhook | OAuth token |
| Manual (forward) | Low | Forward | 1 webhook | Phone mapping | |
| ✈️ Telegram | Manual (forward) | Low | Forward | 1 trigger | chat_id mapping |
| 📮 Email Forwarding | Set once, auto after | Very Low | Semi-auto | 1 IMAP trigger | Email mapping |
| 🌐 Browser Extension | Semi-automatic | Medium | Semi-auto | 1 webhook | Auth token |
📱 Mobile App with Notification Access
A native mobile app that uses a 3-layer hybrid approach to maximize receipt capture: Notification Listener + MediaStore Scanner + Smart Prompt fallback.
🤖 Android
Full notification access via NotificationListenerService API. Combined with MediaStore scanning for full-quality image recovery. Achieves 85%+ full-detail capture rate.
✅ Fully Supported 3-Layer Hybrid🍎 iOS
Apple does NOT allow reading other apps' notifications. Only Share Extension (manual 1-tap) and Shortcuts (limited automation) are possible.
❌ Notification Access Blocked ⚠️ Share Extension OnlyApps Monitored (Android)
| Source App | What Gets Captured | Example |
|---|---|---|
| Shared receipts, payment confirmations | Uber sends receipt via WhatsApp | |
| Gmail / Outlook | Email receipt notifications | Amazon order confirmation email |
| Uber, Lyft, Bolt | Trip receipt notifications | "Your trip receipt: $24.50" |
| DoorDash, UberEats | Food order receipts | Order completed: $22.40 |
| Amazon, eBay | Purchase confirmations | "Your order has shipped" |
| Bank apps | Transaction alerts | "$45.00 charged at Shell" |
| PayPal, Venmo | Payment confirmations | "You paid $30 to John" |
| SMS | Text receipts, bank alerts | "Purchase of $89 at Best Buy" |
⚠️ What Notifications Actually Contain
The NotificationListenerService is sandboxed. It can only read what Android puts IN the notification. It cannot reach back into WhatsApp, Gmail, or any other app to download the original file.
🚫 Notification Access ≠ App Access
CloudCFO cannot open WhatsApp, navigate to a conversation, and download the full receipt image. It can only read the notification bubble content.
┌─────────────────────────────────────────────────────────────────┐ │ WhatsApp Notification Arrives │ ├─────────────────────────────────────────────────────────────────┤ │ │ │ WHAT CLOUDCFO CAN ACCESS: WHAT IT CANNOT ACCESS: │ │ ───────────────────────── ─────────────────────── │ │ │ │ ✅ Title text ❌ Full-resolution image │ │ "Uber" ❌ Original PDF file │ │ ❌ WhatsApp chat history │ │ ✅ Body text ❌ Other messages │ │ "Your trip receipt: $24.50" ❌ App internal storage │ │ ❌ Full email body │ │ ✅ Thumbnail image (compressed) ❌ Email attachments │ │ ~320×320px LOW quality ❌ Any app's database │ │ │ │ ✅ Large icon (small avatar) │ │ ~64×64px │ │ │ │ ✅ Package name (which app sent it) │ │ "com.whatsapp" │ │ │ └─────────────────────────────────────────────────────────────────┘
Image Quality Comparison
✅ Original Receipt (inside app)
1200×2400px — Full quality
All text readable by OCR
Vendor, items, amounts, tax, payment method
✅ OCR success rate: 95%+
❌ Notification Thumbnail
320×320px — Compressed JPEG
Text blurry and unreadable
Details not visible
❌ OCR success rate: ~10%
What CAN Be Extracted from Notification Text Alone
✅ Extractable
Vendor name — from app name or notification title
Amount — "$24.50" parsed from text
Category — inferred from app (Uber = Transport)
Date/time — notification timestamp
Currency — detected from symbol
❌ Not Extractable
Itemized breakdown — line items
Payment method — Visa •1234
Full address / route
Tax details — subtotal vs tax
Receipt image — for record keeping
🔀 Hybrid 3-Layer Capture Approach
Combines three methods to achieve 85%+ full-quality receipt capture with near-zero user effort, and 100% partial capture (vendor + amount always captured).
┌─────────────────────────────────────────────────────────────────────────────┐ │ HYBRID 3-LAYER RECEIPT CAPTURE │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ WhatsApp notification: "Uber receipt: $24.50" + thumbnail │ │ │ │ │ ▼ │ │ ┌─────────────────────────────────────────────────────────────────────────┐│ │ │ LAYER 1: Notification Listener (Automatic — Background) ││ │ │ ││ │ │ • Captures notification TEXT: vendor name, amount, timestamp ││ │ │ • Creates a PARTIAL receipt record with available data ││ │ │ • Gets thumbnail image (low quality — NOT usable for OCR) ││ │ │ • Flags record as "needs_full_image" ││ │ │ • Notes: app=com.whatsapp, timestamp=14:34:00 ││ │ └──────────────────────────┬──────────────────────────────────────────────┘│ │ │ │ │ ▼ │ │ ┌─────────────────────────────────────────────────────────────────────────┐│ │ │ LAYER 2: MediaStore Scanner (Automatic — Background, 2-5s delay) ││ │ │ ││ │ │ • Many apps (WhatsApp, Telegram) auto-save images to phone gallery ││ │ │ • CloudCFO queries Android MediaStore for images saved ±10s of notif ││ │ │ • If FULL QUALITY image found → attach to receipt record ││ │ │ • Upload full image to CloudCFO backend for OCR ││ │ │ • Status → "complete" ││ │ │ ││ │ │ Success rate: ~60% of receipts (apps that save to gallery) ││ │ └──────────────────────────┬──────────────────────────────────────────────┘│ │ │ │ │ FOUND? │ │ ┌────┴────┐ │ │ YES NO │ │ │ │ │ │ ┌───────────┘ └───────────┐ │ │ ▼ ▼ │ │ ┌──────────────────┐ ┌──────────────────────────────────────────────┐ │ │ │ DONE! │ │ LAYER 3: Smart Prompt (Semi-auto — 1 tap) │ │ │ │ │ │ │ │ │ │ Full image │ │ CloudCFO shows notification to user: │ │ │ │ uploaded to │ │ │ │ │ │ backend, OCR │ │ ┌──────────────────────────────────────────┐│ │ │ │ extracts all │ │ │ 💰 CloudCFO detected: Uber $24.50 ││ │ │ │ details. │ │ │ ││ │ │ │ │ │ │ Tap to attach full receipt for ││ │ │ │ User did │ │ │ complete bookkeeping records ││ │ │ │ NOTHING. │ │ │ ││ │ │ │ │ │ │ [Attach Receipt] [Skip — Text Only] ││ │ │ └──────────────────┘ │ └──────────────────────────────────────────┘│ │ │ │ │ │ │ │ If "Attach" → Share picker opens → user │ │ │ │ selects receipt → full quality uploaded │ │ │ │ │ │ │ │ If "Skip" → text-only record kept │ │ │ │ (vendor + amount still captured) │ │ │ └──────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────────────┘
Layer Summary
| Layer | Method | Automatic? | Gets Full Image? | Data Quality | Coverage |
|---|---|---|---|---|---|
| Layer 1 | Notification text extraction | ✅ Fully | ❌ Text + thumbnail only | Partial (vendor + amount) | 100% of receipts |
| Layer 2 | MediaStore gallery scan | ✅ Fully | ✅ Full quality | Complete (OCR works) | ~60% of receipts |
| Layer 3 | Smart prompt (1 tap) | ⚠️ 1 user tap | ✅ Full quality | Complete (user-provided) | ~25% more (of 40% remaining) |
/api/v2/documents/upload with the user's auth token, just like the API channel. The source field is set to "mobile_notification", "mobile_mediastore", or "mobile_share" depending on which layer captured the receipt.🔎 MediaStore Scanner
The secret weapon. Many messaging apps (WhatsApp, Telegram) automatically save received images to the phone's gallery. CloudCFO exploits this to recover full-quality receipt images without any user action.
How It Works
┌─────────────────────────────────────────────────────────────────┐ │ MEDIASTORE SCANNER — AUTOMATIC FULL-IMAGE RECOVERY │ ├─────────────────────────────────────────────────────────────────┤ │ │ │ Apps auto-save images to known gallery paths: │ │ │ │ WhatsApp saves to: │ │ /Android/media/com.whatsapp/WhatsApp/Media/WhatsApp Images/ │ │ │ │ Telegram saves to: │ │ /Telegram/Telegram Images/ │ │ │ │ When notification arrives at 14:34:00: │ │ │ │ 1. CloudCFO notes timestamp: 14:34:00 │ │ 2. CloudCFO notes source app: com.whatsapp │ │ 3. Waits 2-5 seconds (for file to be written to disk) │ │ 4. Queries Android MediaStore: │ │ │ │ SELECT _data, date_added, _size │ │ FROM MediaStore.Images │ │ WHERE date_added >= (14:34:00 - 5 seconds) │ │ AND date_added <= (14:34:00 + 10 seconds) │ │ ORDER BY date_added DESC │ │ LIMIT 1 │ │ │ │ 5. If image found near timestamp: │ │ → Full quality image available! (1200×2400px) │ │ → Upload to CloudCFO backend for OCR │ │ → Receipt record status → "complete" │ │ │ │ 6. If NOT found: │ │ → Trigger Layer 3: Smart Prompt │ │ → Ask user to share manually │ │ │ └─────────────────────────────────────────────────────────────────┘
App Gallery Save Behavior
| App | Auto-Saves to Gallery? | Default Path | MediaStore Works? |
|---|---|---|---|
| ✅ Yes (default ON) | WhatsApp/Media/WhatsApp Images/ |
✅ Yes | |
| Telegram | ✅ Yes (configurable) | Telegram/Telegram Images/ |
✅ Yes |
| Gmail | ❌ No (must download) | N/A — attachments not auto-saved | ❌ No |
| Uber | ❌ No | N/A — in-app only | ❌ No |
| Amazon | ❌ No | N/A — in-app only | ❌ No |
| Bank apps | ⚠️ Some (screenshot) | Screenshots/ |
⚠️ If user screenshots |
| SMS | ✅ MMS images saved | Messages/ |
✅ Yes |
🤖 Android Implementation
Kotlin ▼suspend fun findRecentImage( context: Context, notificationTimestamp: Long, // milliseconds sourcePackage: String ): Uri? { val contentResolver = context.contentResolver // Look for images saved within ±10 seconds of notification val windowStart = (notificationTimestamp / 1000) - 5 val windowEnd = (notificationTimestamp / 1000) + 10 val selection = "${MediaStore.Images.Media.DATE_ADDED} >= ? " + "AND ${MediaStore.Images.Media.DATE_ADDED} <= ?" val selectionArgs = arrayOf( windowStart.toString(), windowEnd.toString() ) val cursor = contentResolver.query( MediaStore.Images.Media.EXTERNAL_CONTENT_URI, arrayOf(MediaStore.Images.Media._ID, MediaStore.Images.Media.DATE_ADDED, MediaStore.Images.Media.SIZE), selection, selectionArgs, "${MediaStore.Images.Media.DATE_ADDED} DESC" ) cursor?.use { if (it.moveToFirst()) { val id = it.getLong( it.getColumnIndexOrThrow(MediaStore.Images.Media._ID) ) return ContentUris.withAppendedId( MediaStore.Images.Media.EXTERNAL_CONTENT_URI, id ) // → Full quality image URI ready for upload! } } return null // No matching image → trigger Layer 3 }
🔐 Required Android Permissions
AndroidManifest.xml ▼<!-- Notification access (Layer 1) --> <uses-permission android:name="android.permission.BIND_NOTIFICATION_LISTENER_SERVICE" /> <!-- MediaStore access (Layer 2) --> <!-- Android 13+ (API 33) --> <uses-permission android:name="android.permission.READ_MEDIA_IMAGES" /> <!-- Android 12 and below --> <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" android:maxSdkVersion="32" /> <!-- Network (upload to CloudCFO) --> <uses-permission android:name="android.permission.INTERNET" />
🔔 Smart Prompt Fallback
When MediaStore scan finds no matching image (apps like Uber, Amazon, Gmail that don't auto-save to gallery), CloudCFO shows a smart notification prompting the user to share the receipt with a single tap.
┌─────────────────────────────────────────────────────────────────┐ │ SMART PROMPT — USER NOTIFICATION │ ├─────────────────────────────────────────────────────────────────┤ │ │ │ Layer 2 MediaStore scan returned: null (no image found) │ │ │ │ │ ▼ │ │ ┌────────────────────────────────────────────────────────────┐ │ │ │ CloudCFO sends its OWN notification: │ │ │ │ │ │ │ │ ┌──────────────────────────────────────────────────────┐ │ │ │ │ │ 💰 CloudCFO 2:34 PM │ │ │ │ │ │ │ │ │ │ │ │ Receipt detected: Uber — $24.50 │ │ │ │ │ │ │ │ │ │ │ │ Tap to attach the full receipt image for │ │ │ │ │ │ complete bookkeeping records. │ │ │ │ │ │ │ │ │ │ │ │ [📎 Attach Receipt] [Skip] │ │ │ │ │ └──────────────────────────────────────────────────────┘ │ │ │ └────────────────────────────────────────────────────────────┘ │ │ │ │ │ ┌────┴─────────┐ │ │ │ │ │ │ Attach Skip │ │ │ │ │ │ ▼ ▼ │ │ ┌──────────┐ ┌──────────────────────────────────────────────┐ │ │ │ Android │ │ Keep text-only record: │ │ │ │ Share │ │ │ │ │ │ Picker │ │ Vendor: Uber │ │ │ │ opens │ │ Amount: $24.50 │ │ │ │ │ │ Date: Jan 8, 2025 │ │ │ │ User │ │ Source: notification_text │ │ │ │ selects │ │ Image: none │ │ │ │ receipt │ │ Status: partial │ │ │ │ │ │ │ │ │ │ Full │ │ Still useful! Vendor + amount captured. │ │ │ │ quality │ │ Can be matched against bank statement later. │ │ │ │ uploaded │ └──────────────────────────────────────────────┘ │ │ └──────────┘ │ │ │ └─────────────────────────────────────────────────────────────────┘
Smart Prompt Behavior
| Feature | Implementation |
|---|---|
| Prompt timing | Shown 5-10 seconds after notification (after MediaStore scan fails) |
| De-duplication | Don't prompt for same vendor twice in 5 minutes |
| Quiet hours | Batch prompts if between 10 PM – 7 AM, show summary in morning |
| Smart grouping | If 3+ receipts pending, show one grouped notification |
| Auto-expire | Remove prompt after 24 hours, keep text-only record |
| User preferences | User can disable prompts entirely (Layers 1+2 still work) |
📈 Realistic Capture Statistics
Expected results for a typical CloudCFO user processing 100 receipts per month.
┌─────────────────────────────────────────────────────────────────┐ │ 100 receipts received on user's phone │ ├─────────────────────────────────────────────────────────────────┤ │ │ │ Layer 1 — Notification Text │ │ Captures basic data from ALL receipts: 100 / 100 (always) │ │ → Vendor + amount extracted ███████████████████ 100% │ │ │ │ Layer 2 — MediaStore Scan │ │ Finds full-quality image automatically: ~60 / 100 │ │ → Full image for OCR ████████████░░░░░░░ 60% │ │ │ │ Layer 3 — Smart Prompt (for remaining 40) │ │ User taps "Attach": ~25 / 40 │ │ User taps "Skip": ~15 / 40 │ │ │ │ ═══════════════════════════════════════ │ │ FINAL RESULTS │ │ ═══════════════════════════════════════ │ │ │ │ Full detail (image + OCR): 85 / 100 │ │ █████████████████░░░ 85% │ │ │ │ Basic data only (vendor + amount): 15 / 100 │ │ ███░░░░░░░░░░░░░░░░░ 15% │ │ │ │ Lost / missed: 0 / 100 │ │ ░░░░░░░░░░░░░░░░░░░░ 0% │ │ │ └─────────────────────────────────────────────────────────────────┘
User Effort Comparison
| Method | User Actions / 100 Receipts | Full-Detail Capture |
|---|---|---|
| Manual forwarding (current) | 500 taps (5 steps × 100) | ~70% (user forgets some) |
| Hybrid 3-Layer (new) | ~25 taps (Layer 3 only, 1 tap each) | 85% full + 15% partial |
| Hybrid with prompts disabled | 0 taps (Layers 1+2 only) | 60% full + 40% partial |
n8n Backend Integration
┌─────────────────────────────────────────────────────────────────────────────┐ │ HOW ALL 3 LAYERS CONNECT TO CLOUDCFO BACKEND │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ EXISTING CHANNELS: NEW MOBILE CHANNELS: │ │ │ │ WhatsApp ────┐ ┌──── Layer 1: mobile_notification │ │ Telegram ────┤ │ (text + thumbnail) │ │ Email ───────┤ │ │ │ API Upload ──┤ ├──── Layer 2: mobile_mediastore │ │ Google Drive─┤ │ (full-quality gallery image) │ │ Gmail OAuth──┤ │ │ │ │ ├──── Layer 3: mobile_share │ │ │ │ (user shared via prompt) │ │ │ │ │ │ ▼ ▼ │ │ ┌─────────────────────────────────────────┐ │ │ │ │ │ │ │ n8n Webhook: /api/v2/documents/upload │ │ │ │ (SINGLE NODE — serves ALL channels) │ │ │ │ │ │ │ │ Body: { │ │ │ │ user_id: "uuid", │ │ │ │ source: "mobile_mediastore", │ │ │ │ file: binary_data, │ │ │ │ metadata: { │ │ │ │ layer: 2, │ │ │ │ original_app: "com.whatsapp", │ │ │ │ notification_text: "Uber $24.50", │ │ │ │ capture_method: "gallery_scan" │ │ │ │ } │ │ │ │ } │ │ │ │ │ │ │ └──────────────────┬──────────────────────┘ │ │ │ │ │ ▼ │ │ Same pipeline: RabbitMQ → OCR → AI → Store → Sync │ │ │ └─────────────────────────────────────────────────────────────────────────────┘
source and metadata.layer fields tell the backend which capture method was used, useful for analytics but not affecting processing.New User Onboarding for Mobile App
1. User installs CloudCFO app from Play Store
Standard app installation, user signs in with CloudCFO credentials.
2. App requests Notification Access permission (Layer 1)
Android redirects to system settings → user toggles CloudCFO ON. Required for Layer 1.
3. App requests Media/Storage permission (Layer 2)
Standard Android permission dialog for READ_MEDIA_IMAGES. Required for MediaStore scanning.
4. PostgreSQL updated
INSERT INTO user_channels (user_id, channel_type, config, active) VALUES ($1, 'mobile_notification', '{"platform":"android","device_id":"...","layers":[1,2,3]}', true)
5. Redis updated
Rate limits initialized: user:{id}:mobile:rate_limit, preferences: user:{id}:mobile:prompt_enabled = true
6. Background service starts — all 3 layers active
Notification listener + MediaStore scanner + Smart prompt. All future receipts captured automatically.
🔍 Auto-Discovery Channels (OAuth)
Fully automatic receipt capture via OAuth authorization. User connects once, CloudCFO monitors forever. Each channel uses 1 shared n8n webhook and stores 1 OAuth token per user in PostgreSQL.
🐘 Shared OAuth Database Schema
PostgreSQL ▼CREATE TABLE user_oauth_tokens ( id SERIAL PRIMARY KEY, user_id UUID REFERENCES users(user_id) NOT NULL, provider VARCHAR(50) NOT NULL, -- 'gmail','outlook','gdrive','dropbox','onedrive','slack','teams' -- OAuth Tokens access_token TEXT NOT NULL, refresh_token TEXT NOT NULL, token_expires_at TIMESTAMP NOT NULL, scopes TEXT[], -- Push Notification / Watch IDs watch_channel_id VARCHAR(255), -- Google Drive / OneDrive watch ID subscription_id VARCHAR(255), -- Graph API subscription ID watch_expires_at TIMESTAMP, -- Must renew periodically change_page_token VARCHAR(255), -- Google Drive changes.list page token -- Metadata created_at TIMESTAMP DEFAULT NOW(), updated_at TIMESTAMP DEFAULT NOW(), last_sync_at TIMESTAMP, status VARCHAR(20) DEFAULT 'active', UNIQUE(user_id, provider) ); CREATE INDEX idx_oauth_provider ON user_oauth_tokens(provider, status); CREATE INDEX idx_oauth_watch ON user_oauth_tokens(watch_expires_at);
Gmail Auto-Discovery
OAuth + Pub/Sub ▼Monitors user's Gmail inbox for receipt emails from known senders (Uber, Amazon, airlines, etc.). Fully automatic after one-time OAuth consent.
Complete Process Flow
1. Consent Flow (One-time)
User clicks "Connect Gmail" in CloudCFO dashboard → Redirected to Google OAuth consent screen → User approves "Read your Gmail messages" scope → Google returns authorization code.
2. Token Exchange
CloudCFO backend exchanges auth code for access_token + refresh_token. Tokens are stored in PostgreSQL user_oauth_tokens table with provider='gmail'.
3. PostgreSQL Update
INSERT INTO user_oauth_tokens (user_id, provider, access_token, refresh_token, token_expires_at, scopes) — stores user's Gmail credentials securely.
4. Set Up Gmail Watch (Push Notification)
CloudCFO calls POST gmail.googleapis.com/gmail/v1/users/me/watch with the user's token. Topic: projects/cloudcfo/topics/gmail-notifications. This tells Google: "notify my webhook when this user gets new email."
5. Redis Cache
Initialize user:{id}:gmail:last_history_id to track which emails have been processed. Rate limits set.
6. Push Notifications Arrive (Automatic, forever)
When user receives email → Google Pub/Sub sends notification to CloudCFO's single webhook → n8n identifies user by email → fetches email using stored OAuth token → checks if receipt → processes.
New email arrives in user's Gmail inbox │ ▼ ┌────────────────────────────────────────────────────────────┐ │ Google Pub/Sub → CloudCFO Webhook │ │ │ │ POST https://api.cloudcfo.ai/webhooks/gmail │ │ Body: { │ │ "message": { │ │ "data": "base64({emailAddress: user@gmail.com})" │ │ } │ │ } │ └────────────────────────┬───────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ n8n Webhook Node (SINGLE NODE for ALL Gmail users) │ │ │ │ 1. Decode payload → get user@gmail.com │ │ 2. Query PostgreSQL: SELECT access_token │ │ FROM user_oauth_tokens │ │ WHERE provider='gmail' │ │ AND user_id = (SELECT user_id FROM users │ │ WHERE email = 'user@gmail.com') │ │ 3. If token expired → refresh using refresh_token │ │ 4. Call Gmail API with token to fetch new emails │ │ 5. Filter for receipt-like emails (known senders list) │ │ 6. Extract attachments / content │ │ 7. Send to RabbitMQ processing queue │ └────────────────────────────────────────────────────────────┘
Outlook / Microsoft 365 Auto-Discovery
Graph API + Subscriptions ▼Monitors user's Outlook inbox via Microsoft Graph API change notifications. Same pattern as Gmail but using Microsoft's ecosystem.
1. Consent Flow
User clicks "Connect Outlook" → Microsoft OAuth consent screen → Approves Mail.Read scope → Auth code returned.
2. Token Exchange + PostgreSQL
Exchange code for tokens → INSERT INTO user_oauth_tokens (..., provider='outlook', ...)
3. Create Graph API Subscription
POST graph.microsoft.com/v1.0/subscriptions — Subscribe to /me/mailFolders/inbox/messages with changeType created. Notification URL: https://api.cloudcfo.ai/webhooks/outlook
4. Store subscription_id in PostgreSQL
Graph API returns a subscription ID — stored for renewal (subscriptions expire in 3 days for mail).
5. Notifications Arrive Automatically
New email → Microsoft sends change notification to webhook → n8n identifies user by subscription_id → fetches email with token → processes receipt.
New email arrives in user's Outlook inbox │ ▼ ┌────────────────────────────────────────────────────────────┐ │ Microsoft Graph API → CloudCFO Webhook │ │ │ │ POST https://api.cloudcfo.ai/webhooks/outlook │ │ Body: { │ │ "value": [{ │ │ "subscriptionId": "sub-abc-123", │ │ "resourceData": { "id": "message-xyz" } │ │ }] │ │ } │ └────────────────────────┬───────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ n8n Webhook Node (SINGLE NODE) │ │ │ │ 1. Extract subscriptionId → lookup user_id in DB │ │ 2. Get user's OAuth token │ │ 3. Fetch email: GET graph.microsoft.com/.../messages/{id} │ │ 4. Check if receipt → Process │ └────────────────────────────────────────────────────────────┘
PATCH .../subscriptions/{id} to renew. More frequent than Gmail's 7-day cycle.Google Drive Auto-Discovery
OAuth + Changes Watch API ▼Monitors the user's entire Google Drive for new receipt-like files. User drops receipts into any folder — via phone, desktop, or web — and CloudCFO auto-detects and processes them. No dedicated folder needed.
1. OAuth Consent
User clicks "Connect Google Drive" → Google OAuth consent screen → Approves drive.readonly scope → Auth code returned → Tokens stored in PostgreSQL.
2. Get Start Page Token
Call GET drive.googleapis.com/drive/v3/changes/startPageToken to get the current state marker. Store in PostgreSQL: change_page_token column. This ensures CloudCFO only processes files added after onboarding, not the user's entire history.
3. Set Up Changes Watch (Entire Drive)
POST drive.googleapis.com/drive/v3/changes/watch with unique channel_id: "watch-user-{uuid}" and address: https://api.cloudcfo.ai/webhooks/gdrive. This watches all changes across the user's entire Drive — no folder restriction.
4. PostgreSQL Update
Store watch_channel_id, watch_resource_id, watch_expires_at, and change_page_token in user_oauth_tokens.
5. New Files Detected Automatically
User saves any file anywhere in Drive → Google sends push notification → n8n webhook receives it → calls changes.list to get changed files → filters for receipt-like MIME types → downloads and processes matching files.
User saves receipt.pdf to ANY folder in Google Drive (phone upload, browser download, desktop sync — any method) │ ▼ ┌────────────────────────────────────────────────────────────┐ │ Google Drive Changes API → Push Notification │ │ │ │ POST https://api.cloudcfo.ai/webhooks/gdrive │ │ Headers: │ │ X-Goog-Channel-ID: watch-user-abc123 ← Identifies user │ │ X-Goog-Resource-State: change │ │ │ │ Note: Webhook only says "something changed" — │ │ not WHICH file. Must call changes.list to find out. │ └────────────────────────┬───────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ n8n Webhook Node (SINGLE NODE for ALL Google Drive users) │ │ │ │ 1. Extract Channel-ID → lookup user_id + token in DB │ │ 2. Get stored change_page_token from PostgreSQL │ │ 3. Call: GET changes/list?pageToken={token} │ │ → Returns list of changed files since last check │ │ 4. FILTER: Only process receipt-like files │ │ ✅ image/jpeg, image/png, image/webp │ │ ✅ application/pdf │ │ ❌ application/vnd.google-apps.document (ignore) │ │ ❌ application/vnd.google-apps.spreadsheet (ignore) │ │ ❌ application/vnd.google-apps.presentation (ignore) │ │ ❌ Folders, shortcuts, forms (ignore) │ │ ❌ Files < 10KB or > 25MB (ignore) │ │ 5. Download matching files via Drive API │ │ 6. Store in MinIO → send to RabbitMQ pipeline │ │ 7. Update change_page_token in PostgreSQL for next poll │ └────────────────────────────────────────────────────────────┘
Smart Backend Filtering
Since we watch the entire Drive, CloudCFO receives notifications for all file activity (edits, shares, moves, etc.). The n8n node filters aggressively to only process actual receipts:
| Check | Rule | Reason |
|---|---|---|
| ✅ MIME type | Only image/* and application/pdf |
Receipts are images or PDFs, not Google Docs |
| ✅ New files only | createdTime == modifiedTime |
Ignore edits to existing files — only new uploads |
| ✅ File size | Between 10KB and 25MB | Skip tiny icons and huge non-receipt files |
| ✅ Not trashed | trashed == false |
Skip deleted files |
| ⚡ De-duplication | Check file hash against processed files in DB | Skip if same receipt already processed from another channel |
changes.watch channels expire after a maximum of 7 days. A scheduled n8n cron job renews all active watches weekly. Still only 1 n8n node — queries all users from DB and renews in a loop.How Users Drop Receipts
With entire-drive monitoring, every method "just works":
| Method | How It Works | Detected? |
|---|---|---|
| 📱 Phone camera | Take photo of paper receipt → auto-syncs to Google Photos/Drive | ✅ Auto |
| 📱 Phone "Save to Drive" | In any app, tap Share → Save to Google Drive (any folder) | ✅ Auto |
| 💻 Browser download | Download PDF receipt → drag to Google Drive in browser | ✅ Auto |
| 🖥️ Desktop sync | Save receipt to Google Drive folder on desktop (Drive for Desktop app) | ✅ Auto |
| 📧 Gmail attachment | Open email receipt → "Save to Drive" button in Gmail | ✅ Auto |
| 📸 Google Lens scan | Scan receipt with Google Lens → Save to Drive | ✅ Auto |
🔧 changes.watch Setup & changes.list Processing
API Flow ▼// ─── STEP 1: Get starting page token (during onboarding) ─── GET https://www.googleapis.com/drive/v3/changes/startPageToken Authorization: Bearer {user_access_token} Response: { "startPageToken": "4285" } → Store "4285" in PostgreSQL: user_oauth_tokens.change_page_token // ─── STEP 2: Create changes.watch channel ─── POST https://www.googleapis.com/drive/v3/changes/watch?pageToken=4285 Authorization: Bearer {user_access_token} Content-Type: application/json { "id": "watch-user-abc123-uuid", "type": "web_hook", "address": "https://api.cloudcfo.ai/webhooks/gdrive", "token": "user_id=abc123" // custom token for user lookup } Response: { "kind": "api#channel", "id": "watch-user-abc123-uuid", "resourceId": "abc_resource_123", "expiration": 1735689600000 // ~7 days max } → Store watch_channel_id, watch_resource_id, expiration in DB // ─── STEP 3: When webhook fires (file changed anywhere) ─── Google sends to https://api.cloudcfo.ai/webhooks/gdrive: Headers: X-Goog-Channel-ID: watch-user-abc123-uuid X-Goog-Channel-Token: user_id=abc123 X-Goog-Resource-State: change // ─── STEP 4: n8n fetches changed files ─── GET https://www.googleapis.com/drive/v3/changes?pageToken=4285&fields=* Authorization: Bearer {user_access_token} Response: { "changes": [ { "fileId": "1sP77lasbdEw...", "removed": false, "file": { "name": "uber-receipt-jan.pdf", "mimeType": "application/pdf", // ✅ Receipt-like "size": "245000", // ✅ 245KB — valid range "createdTime": "2025-01-08T14:34:00Z", "trashed": false // ✅ Not trashed } }, { "fileId": "8xQ12kfaMnb...", "removed": false, "file": { "name": "Project Notes", "mimeType": "application/vnd.google-apps.document" // ❌ Skip } } ], "newStartPageToken": "4290" // → Save for next poll } // ─── STEP 5: Download only the receipt-like file ─── GET https://www.googleapis.com/drive/v3/files/1sP77lasbdEw...?alt=media Authorization: Bearer {user_access_token} → Binary PDF data → Store in MinIO → Send to RabbitMQ
Dropbox Auto-Discovery
OAuth + Webhooks ▼Dropbox sends a webhook whenever files change in user's account. CloudCFO monitors a specific folder path.
1. OAuth Consent
User approves Dropbox OAuth → tokens stored. Scope: files.metadata.read files.content.read
2. Register Webhook
In Dropbox App Console, set webhook URL to https://api.cloudcfo.ai/webhooks/dropbox. One URL serves all users.
3. Create Folder
Use user's token to create /CloudCFO Receipts folder in their Dropbox.
4. Store Cursor
Call /files/list_folder to get initial cursor. Store in Redis: user:{id}:dropbox:cursor
5. Changes Detected
Dropbox webhook sends list of account_ids with changes → n8n maps account_id to user → uses cursor to get new files → downloads and processes.
User saves file to /CloudCFO Receipts in Dropbox │ ▼ ┌────────────────────────────────────────────────────────────┐ │ Dropbox Webhook │ │ │ │ POST https://api.cloudcfo.ai/webhooks/dropbox │ │ Body: { │ │ "list_folder": { │ │ "accounts": ["dbid:AAB123..."] ← User's account ID │ │ } │ │ } │ └────────────────────────┬───────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ n8n Webhook Node (SINGLE NODE) │ │ │ │ 1. For each account_id → lookup user_id + token in DB │ │ 2. Get cursor from Redis │ │ 3. Call /files/list_folder/continue with cursor │ │ 4. Download new files in /CloudCFO Receipts │ │ 5. Update cursor in Redis │ │ 6. Process receipts │ └────────────────────────────────────────────────────────────┘
OneDrive Auto-Discovery
Graph API + Subscriptions ▼Same Microsoft Graph API pattern as Outlook, but subscribing to drive changes instead of mail.
1. OAuth Consent
User approves Files.Read scope via Microsoft OAuth.
2. Create Folder + Subscribe
Create "CloudCFO Receipts" folder → POST graph.microsoft.com/v1.0/subscriptions on /me/drive/root with changeType updated.
3. PostgreSQL + Redis
Store subscription_id and delta link (cursor) for tracking changes.
4. Changes Detected
File added → Graph API sends notification → n8n maps subscription to user → uses delta API to find new files → downloads and processes.
Slack Auto-Discovery
OAuth + Events API ▼Monitors Slack channels/DMs where receipts are shared (e.g., #expenses channel). Real-time event-driven.
1. OAuth Consent
User installs CloudCFO Slack app → approves channels:history, files:read scopes → workspace token stored.
2. Events API
Slack Events API configured with URL https://api.cloudcfo.ai/webhooks/slack. Subscribes to message events.
3. PostgreSQL
Store workspace token, team_id, user_id mapping.
4. Messages with files auto-captured
User shares receipt image in any authorized channel → Slack sends event → n8n checks for file attachments → downloads with token → processes.
Microsoft Teams Auto-Discovery
Graph API + Subscriptions ▼Monitors Teams chats/channels for shared receipt files. Same Graph API pattern as Outlook/OneDrive.
1. OAuth Consent
User approves Chat.Read, ChannelMessage.Read.All scopes.
2. Create Subscription
POST graph.microsoft.com/v1.0/subscriptions on /me/chats/getAllMessages.
3. PostgreSQL
Store subscription_id, tokens.
4. Messages with attachments auto-captured
File shared in chat → Graph API notification → n8n identifies user → downloads file → processes.
↗️ Forwarding Channels (Manual / Semi-Automatic)
For platforms that don't support OAuth auto-discovery (WhatsApp, Telegram, Signal, WeChat), CloudCFO provides a shared account/number where users forward receipts. User identification is based on sender ID.
user_channels table.WhatsApp Forwarding
Manual Forward → Auto ID ▼CloudCFO operates ONE WhatsApp Business number. All 1,000+ users forward receipts to the same number. Sender's phone number identifies who they are.
Architecture
User receives receipt on their personal WhatsApp │ │ User FORWARDS to +1-888-CLOUDCFO (shared number) ▼ ┌────────────────────────────────────────────────────────────┐ │ WhatsApp Business API Webhook │ │ │ │ POST https://api.cloudcfo.ai/webhooks/whatsapp │ │ { │ │ "entry": [{ │ │ "changes": [{ │ │ "value": { │ │ "messages": [{ │ │ "from": "+15551234567", ← Sender's phone number│ │ "type": "image", │ │ "image": { "id": "media_abc123" } │ │ }] │ │ } │ │ }] │ │ }] │ │ } │ └────────────────────────┬───────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ n8n Webhook Node (SINGLE NODE) │ │ │ │ 1. Extract sender phone: +15551234567 │ │ 2. SELECT user_id FROM user_channels │ │ WHERE channel_type = 'whatsapp' │ │ AND config->>'phone' = '+15551234567' │ │ 3. If not found → Reply "Please register first" │ │ 4. If found → Download media → Process with user_id │ │ 5. Reply: "✅ Receipt captured: Uber $24.50" │ └────────────────────────────────────────────────────────────┘
New User Registration for WhatsApp
1. User registers on CloudCFO (web/app)
Provides phone number during signup.
2. PostgreSQL Updated
INSERT INTO user_channels (user_id, channel_type, config) VALUES ($1, 'whatsapp', '{"phone":"+15551234567"}')
3. Verification (optional)
CloudCFO sends WhatsApp message to user: "Reply YES to confirm this number." Prevents spoofing.
4. Ready
User can now forward any receipt to the CloudCFO number. Identified automatically by phone.
INSERT INTO user_channels (user_id, channel_type, config, active) VALUES ( 'user-uuid-123', 'whatsapp', '{"phone": "+15551234567", "verified": true, "registered_at": "2025-01-08"}', true );
Telegram Bot Forwarding
Manual Forward → Auto ID ▼CloudCFO runs ONE Telegram Bot (@CloudCFO_Bot). Users forward receipts to the bot. Identified by chat_id.
User forwards receipt to @CloudCFO_Bot │ ▼ ┌────────────────────────────────────────────────────────────┐ │ Telegram Bot API Webhook │ │ │ │ { │ │ "message": { │ │ "chat": { "id": 123456789 }, ← User's chat_id │ │ "from": { "id": 123456789 }, │ │ "document": { │ │ "file_id": "BAADBAADxx...", │ │ "file_name": "receipt.pdf" │ │ } │ │ } │ │ } │ └────────────────────────┬───────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ n8n Telegram Trigger Node (SINGLE NODE) │ │ │ │ 1. Extract chat_id: 123456789 │ │ 2. SELECT user_id FROM user_channels │ │ WHERE channel_type = 'telegram' │ │ AND config->>'chat_id' = '123456789' │ │ 3. Download file via Telegram API │ │ 4. Process with user_id │ │ 5. Reply in chat: "✅ Receipt captured!" │ └────────────────────────────────────────────────────────────┘
Auto-Registration via /start
Telegram is unique — users can self-register by chatting with the bot:
1. User sends /start to @CloudCFO_Bot
Bot receives chat_id automatically from Telegram.
2. Bot asks for email
"Welcome! Please enter your CloudCFO email to link your account."
3. PostgreSQL Updated
INSERT INTO user_channels (user_id, 'telegram', '{"chat_id":"123456789"}')
4. Ready
All future messages with documents are auto-processed.
Email Forwarding
Semi-Automatic ▼CloudCFO provides ONE inbox: receipts@cloudcfo.ai. Users can forward manually or set up auto-forwarding rules. Two identification methods available.
Method A: Sender Email
User forwards from their registered email. Identified by From: header.
From: john@company.com ← Lookup key
To: receipts@cloudcfo.ai
Subject: Fwd: Your Uber ReceiptMethod B: Plus-Addressing
Each user gets a unique address like receipts+abc123@cloudcfo.ai. More reliable.
To: receipts+abc123@cloudcfo.ai
↑
User's unique code (assigned at registration)🖥️ VPS Services — Updates Per New User
What databases and services on your two VPS servers need to be updated when a new user signs up.
Main VPS — n8n.cloudcfo.ai
RAG VPS — rag.cloudcfo.ai
Summary
⚙️ Automated User Provisioning
Yes, the entire process can be fully automated. A single API call provisions everything.
┌─────────────────────────────────────────────────────────────────────┐ │ Automated Provisioning Pipeline │ │ │ │ User clicks "Sign Up" on cloudcfo.ai │ │ │ │ │ ▼ │ │ POST /api/v1/users/provision │ │ { email, name, phone, plan, channels[] } │ │ │ │ │ ├──► PostgreSQL │ │ │ ├── INSERT users │ │ │ ├── INSERT user_subscriptions │ │ │ ├── INSERT user_channels (per selected channel) │ │ │ └── INSERT api_credentials │ │ │ │ │ ├──► Redis │ │ │ ├── SET rate limits │ │ │ ├── SET session data │ │ │ └── SET preference defaults │ │ │ │ │ ├──► MinIO │ │ │ ├── CREATE bucket user-{uuid} │ │ │ └── CREATE folders /receipts /invoices /reports │ │ │ │ │ ├──► Baserow │ │ │ └── CREATE user record in KB management │ │ │ │ │ ├──► Qdrant (optional) │ │ │ └── CREATE collection or namespace │ │ │ │ │ └──► OAuth Channels (async, user-initiated) │ │ ├── Gmail: Wait for user to click "Connect" │ │ ├── Drive: Wait for user to click "Connect" │ │ └── Each stores tokens on completion │ │ │ │ Response: { user_id, api_key, endpoints, bucket } │ │ │ │ │ ▼ │ │ Welcome email sent with credentials │ │ │ └─────────────────────────────────────────────────────────────────────┘
rag.cloudcfo.ai handles it perfectly — deterministic steps, no AI reasoning needed. The provisioning service runs as a Docker container alongside your existing services. An n8n workflow can also orchestrate this via HTTP Request nodes.✅ n8n Node Architecture Confirmation
Final confirmation of how many n8n nodes are needed and what changes per user.
| Channel | n8n Node Type | Count | Created When? | Changes Per User? |
|---|---|---|---|---|
| 📱 Mobile App | Webhook | 1 | Platform setup (once) | ❌ Never |
| 📧 Gmail | Webhook (Pub/Sub) | 1 | Platform setup (once) | ❌ Never |
| 📬 Outlook | Webhook (Graph API) | 1 | Platform setup (once) | ❌ Never |
| 📁 Google Drive | Webhook | 1 | Platform setup (once) | ❌ Never |
| 📦 Dropbox | Webhook | 1 | Platform setup (once) | ❌ Never |
| ☁️ OneDrive | Webhook (Graph API) | 1 | Platform setup (once) | ❌ Never |
| 💬 Slack | Webhook (Events API) | 1 | Platform setup (once) | ❌ Never |
| 🟦 Teams | Webhook (Graph API) | 1 | Platform setup (once) | ❌ Never |
| Webhook | 1 | Platform setup (once) | ❌ Never | |
| ✈️ Telegram | Telegram Trigger | 1 | Platform setup (once) | ❌ Never |
| 📮 Email Fwd | IMAP Trigger | 1 | Platform setup (once) | ❌ Never |
| 🌐 Browser Ext. | Webhook | 1 | Platform setup (once) | ❌ Never |
| TOTAL | ~12 | All at platform setup | ❌ NEVER | |
🎯 Bottom Line
User #1 and User #10,000 use the exact same n8n workflows. The only difference is a few rows in PostgreSQL and Redis. No n8n nodes are ever created, modified, or deleted when users sign up or leave. The multi-tenant architecture scales infinitely through database-level user isolation.