Automatic Rescraping: Keep Your Website Chatbot Fresh
Your website changes. Your prices, FAQs, and blog posts evolve. If your chatbot doesn't update with them, answers get stale - and visitors bounce.
With Automatic Rescraping in Ultimo Bots, your assistant keeps learning from the exact pages you select, on a schedule you control. Fresh content in, accurate answers out.
Why Automatic Rescraping Matters
- Always up-to-date answers - New copy, new products, or revised policies are reflected in the bot without manual retraining.
- Fewer support tickets - Clear, current responses reduce repetitive questions and handovers.
- Better user signals for SEO - Helpful, accurate answers keep visitors engaged longer and reduce pogo-sticking back to search.
- No maintenance headache - Set your schedule once and focus on content; the bot follows.
What You Can Configure
Our new settings give you precision control over what gets refreshed and when:
1) Frequency
Choose how often Ultimo Bots should revisit your content:
- Daily - Great for fast-moving sites (blogs, news, product catalogs, pricing pages).
- Monthly - Perfect for static sites or documentation that changes occasionally.
2) Scan for New URLs During Rescraping
Toggle Fetch New Links to let the crawler discover new internal pages during each run (e.g., new blog posts or help articles) and add them to your assistant’s knowledge - up to your plan’s content allowance.
3) Select Links (Granular Control)
Pick exactly which pages to include:
- Search box to quickly find URLs.
- Select All or cherry-pick specific pages.
- Deselect pages you don’t want rescraped (e.g., legacy or seasonal pages).
Result: you refresh only what matters - keeping quality high and costs predictable.
How It Works (In Plain English)
On the schedule you choose, Ultimo Bots:
- Visits the selected URLs (and, if enabled, follows new internal links).
- Extracts clean text and structure.
- Updates your assistant’s knowledge so the next visitor gets the latest information.
No code. No manual re-uploads. Just accurate answers.
Best-Practice Setups
- Content teams publishing weekly or daily
Use Daily + Fetch New Links for blogs and help centers. Keep cornerstone pages always selected. - SMBs with steady websites
Use Monthly without link discovery. Select key pages (Home, Pricing, Services, FAQ, Contact). - Product or policy changes
Temporarily switch to Daily for launch weeks, then revert to Monthly.
Tips for Cleaner, Higher-Quality Answers
- Keep page titles and headings clear; your assistant mirrors your structure.
- Consolidate duplicate pages to avoid mixed signals.
- Pin critical URLs in Select Links so they’re always refreshed.
FAQ
Will rescraping overwrite my existing training?
It updates the knowledge with the latest version of each selected page. Your assistant stays aligned to your live site.
Does it respect my plan limits?
Yes. New pages discovered via Fetch New Links are added up to your content allowance. You’re always in control via Select Links.
Can I turn it off or change the schedule?
Any time. Switch between Daily/Monthly, toggle discovery, or adjust the URL list in seconds.
What if a page is removed on my site?
When a URL disappears, the assistant stops learning from it on subsequent runs.
Automatic Rescraping gives you continuous accuracy with zero busywork - so your visitors get reliable answers, you save support time, and your site engagement lifts.