Manual Website Re-Scan + Automatic Change Detection

Problem

Currently, website scan happens only once during business creation.

If the website later:

  • Adds new pages

  • Updates existing pages

  • Changes structure

There is no way to re-scan.

The only option is:
Delete business → Add again → Full re-scan

This causes:

  • Credits wasted on already scanned pages

  • Loss of historical data

  • Poor user experience

  • Not scalable for growing websites


What We Need

1️⃣ Manual Incremental Re-Scan

Add a “Re-Scan Website” button inside each Business.

When user clicks Re-Scan:

  1. Crawl website again

  2. Compare with previously scanned pages

  3. Detect:

  • ✅ New pages → Scan + deduct credits

  • 🔁 Existing pages → Skip (no credits)

  • 📝 Changed pages (optional) → Scan only if needed

  • ❌ Deleted pages → Mark as inactive

Important:

  • Do NOT delete existing data

  • Do NOT consume credits for already scanned pages

  • Show how many credits will be used before confirming

Example UI message:

We found 5 new pages.
Credits required: 5
Do you want to continue?


2️⃣ Automatic Change Detection

Add option to automatically detect website changes.

How it works:

System periodically checks:

  • Sitemap

  • Page URLs

  • Content changes

If new pages are detected:

Option A:
Notify user:

We found 3 new pages. Scan now?

Option B:
If Auto-Scan is enabled → scan automatically and deduct credits.


Business Settings Additions

Under Website Settings:

  • Enable Auto-Scan for New Pages

  • Scan Frequency: Weekly / Monthly

  • Auto-scan changed pages: Yes / No


Acceptance Criteria

  • User can re-scan without deleting business

  • Only new pages consume credits

  • Existing pages remain untouched

  • Historical data stays intact

  • Clear credit preview before scan

  • Auto-scan toggle works

  • System logs scan timestamp per page


Why This Matters

  • Saves credits

  • Protects user data

  • Supports growing websites

  • Reduces churn

  • Makes ClickGrow scalable

Please authenticate to join the conversation.

Upvoters
Status

Completed

Board
💡

Feature Request

Tags

Brand Intelligence

Date

4 months ago

Author

Vinoth Prabakar K

Subscribe to post

Get notified by email when there are changes.