Python Yahoo Finance scrapers risk legal liability for Taiwan stock data
A tutorial showing how to scrape Taiwan Stock Exchange prices from Yahoo Finance TW is making rounds in developer circles. The five-minute Python script uses BeautifulSoup to extract real-time prices for TWSE-listed stocks. The approach is straightforward: fake a browser user-agent, parse meta tags, return a dictionary.
The problem: it's likely illegal under TWSE's trading information regulations.
TWSE prohibits unauthorized retransmission or commercialization of its market data. Fubon Securities explicitly warns that violations carry civil and criminal liability. Yahoo Finance TW's terms of service also forbid automated scraping. The risk isn't theoretical: approved data vendors exist precisely because TWSE enforces these restrictions.
What the official options look like
TWSE operates an official OpenAPI at openapi.twse.com.tw, though developers describe it as "clunky." For real-time data, the exchange requires either direct connections or approved vendor relationships. Fubon offers Web and WebSocket APIs sourced from providers like Infotimes and Fugle. iTick positions itself as a middle ground, offering free plans for prices, volumes, and capital flows through Python and Java SDKs.
The vendor route costs money, but it addresses three problems: legal compliance, stable data feeds, and rate limit management. Yahoo Finance scrapers face all three issues. The site changes its HTML structure regularly, breaking parsers. Rate limiting kicks in after moderate request volumes. And the legal exposure grows with any commercial use.
The trade-off calculation
For personal portfolio tracking, a simple scraper might survive under the radar. For anything touching production systems, client data, or automated trading, the risk profile changes. TWSE lists over 900 stocks. At scale, the reliability gap between scraping and official feeds widens.
The tutorial's promise of "real-time" data also misleads. Yahoo Finance TW introduces delays compared to direct exchange feeds. For quantitative trading or enterprise applications where milliseconds matter, WebSocket connections to approved vendors deliver actual real-time prices. REST polling adds latency that compounds across hundreds of symbols.
The open-source ecosystem offers some alternatives: Rust's twstock library and R's TWSE GitHub repo (seven stars, limited maintenance). These face the same underlying constraint: without TWSE approval, they're scraping or using intermediaries with the same legal questions.
What this means in practice
If you're building anything that matters, pay for proper access or use the official API despite its limitations. The cost of vendor feeds is insurance against both technical fragility and legal exposure. TWSE's restrictions aren't arbitrary: the exchange controls data distribution as part of market integrity rules.
The "five-minute scraper" solves a real developer frustration with TWSE's API quality. But quick isn't the same as right. The fine print matters here.