Browser Automation for Scientific Data Collection

Access scientific databases that have no REST API by controlling a real browser programmatically. Uses the browser-use framework (vision-based LLM browser automation).

When to Use

• Target database has no REST API (e.g., GEPIA2, some COSMIC pages)
• curl returns 403/captcha/login required and the data is publicly viewable in a browser
• Need to navigate multi-step web forms (e.g., TIMER2.0 correlation analysis)
• Need to download files from web interfaces (e.g., GEO supplementary data)
• API exists but is severely rate-limited and web access is faster

When NOT to use:

• REST API is available and working → use curl
• Data requires paid subscription → do not circumvent paywalls
• Data can be obtained from an alternative open API → prefer the API