Automating web content collection

Communications

Wiker is the first local network to connect people and organizations in small and medium-sized towns through information about their economic, cultural and social activities. To do this, Wiker gathers relevant local information from numerous websites and brings it together in a single service. This collection process is time-consuming and prone to errors. Each team member spends an average of half an hour a day checking the websites they follow, retrieving useful information and labeling it.

uc communication

Modules used

From the web to standardized content in a database

sc modules 1sc modules 2

The solution

The SmartMyData platform automated the entire process. The scraping worker retrieves information from the relevant websites. The function worker cleans and formats the data, then the Prediction worker labels the records. An INSEE module adds INSEE codes to the municipalities, and finally the Database worker saves the records to the database.

Source data and workflow

  • Mapping
  • API
  • Functions
  • Web collection
  • Prediction

Return on investment

Automated data collection and labeling, while minimizing the risks associated with manual work.

  • Automated information collection from a wide variety of sources
  • Automatic record labeling using a machine learning module
  • Team members freed up for more rewarding tasks
  • Reduced risks associated with manual work

Let’s talk about your project!

Contact us