About the Universities500 reader
Universities500Bot is the program that reads public scholarship pages and public job feeds so students can search scholarships and internships in one place. It is run by Universities500. It does not use AI to read pages, and it does not log in, fill in forms or read anything behind a sign-in.
What it does
- Reads scholarship pages about once every two days, and employers' public job-board feeds about once a day.
- Reads robots.txt first and obeys it, including any Crawl-delay.
- Waits at least three seconds between requests to the same website, and reads no more than about 35 pages per university in a run.
- Identifies itself as
Universities500Botand links to this page in its User-Agent. - Copies facts (for a scholarship: name, value, deadline, eligibility; for an internship: title, company, city) and links back to your page. It does not copy whole pages or job descriptions.
How to stop it
Add this to your robots.txt and it will stop on the next run:
User-agent: Universities500Bot
Disallow: /To remove or correct an entry about your university, change the official page it links to. The next run picks the change up.