Crawly
The web crawler & data indexer!
The web crawler & data indexer!
Create data instance that fits your favorite website using XPath and scrape the data from it for your further analysis of the scraped data.
Each data instance allows you to use crawler and indexing techniques to be able to use the data for searching your query. You can always merge data with an already indexed one to make it easy to update your search data pool.
Use one of the supported query models to search with your specific query. E.g. TF-IDF vector-based model or boolean model for simple text presence checking.
Crawly is easy to use web scraper where you can create your data instance that servers you as a crawler, data indexation with the ability to query and search the scraped data. You can fit it for almost any website using XPath.
Create data instances for each web you want to track and have an easy search.
The simple graphical user interface will help you to set and use the app very fastly.
Choose query model that fits your search the best to get the most appropriate result you wanted.
Web scraping itself is not illegal, but people need to be careful with how to use this technique even though there are still a lot of grey areas around law enforcement of web scraping. Remember that it is possible that the owners of websites can sue you for any reason. If you don't respect the rules.