Member-only story

Python HOW: Migrate Prodigy Database to MySQL or PostgreSQL Using Python

3 min readOct 13, 2020

Learn how to save Prodigy annotations in a remote database for collaborative annotating

Photo edited by the author from prodi.gy

Prodigy is developed by Explosion AI, the folks behind spaCy, so it integrates with it organically 🍻. This article describes in detail how to migrate the default local SQLite database schema into a remote MySQL or PostgreSQL database

TL;DR: full code

Default Database 📍

The first time you run Prodigy, it will create a folder names .prodigy in your home directory with 2 files:

By default, Prodigy looks for its configuration in prodigy.json which has the default database "db": "sqlite”. So, by default, annotations are saved in the SQLite database prodigy.db. This is great to start labelling your data straight away without having to worry about where to save them

Remote Server 🌍

If the task requires multiple annotators, or you want to save your annotations on a remote MySQL or PostgreSQL server, you…

Thanks a lot for the interesting article. I have two questions though.

1. Does the migration includes the database schema and data or is it only the schema? It is not super clear from the article.

2. How would it be different when I already have the…

Thanks, Gabriel. Great article.

To suggest one change: For Postgres, we need to add one more line before creating tables — db.bind(models.values()) — apparently we need to re-bind the models to the postgres database.

Check more here— https://github.com/coleifer/peewee/issues/2482

Python HOW: Migrate Prodigy Database to MySQL or PostgreSQL Using Python

Default Database 📍

Remote Server 🌍

Create an account to read the full story.

Written by Gabriel Harris Ph.D.

Responses (2)

More from Gabriel Harris Ph.D.

Python HOW: Using Poetry, Make, and pre-commit-hooks to Setup a Repo Template for your Team

Bring consistency, rigour, and best practices to your messy data science team

Python HOW: Making Facebook API calls Using facebook-sdk

At the end of this article you will understand how to make Graph API calls on a Facebook page you’re managing using the facebook-sdk…

Python HOW: create a local pre-commit hook for conda

Define a local pre-commit hook that runs a local python script to automatically exporting your conda environment YAML file

Python HOW: Create requirements.txt Using pipenv

A requirements.txt file is a snapshot of all the packages you’ve used. You will definitely need one to build a Docker image!

Recommended from Medium

5 Powerful F-String Tricks Every Python Developer Should Know!

Learn five powerful f-string techniques to write cleaner, faster, and more readable Python code.

Build Your Dream Python SaaS with 5 Best Open Source: Cost-Effective, Full-Featured!

Are you a developer looking to build a robust and efficient Python-based Software as a Service (SaaS) application?

Lists

Coding & Development

Predictive Modeling w/ Python

General Coding Knowledge

Practical Guides to Machine Learning

Cron Jobs with PostgreSQL and the pg_cron extension

plus Docker and Ubuntu

Loops in Python {1}

Hello,

Mastering Celery: A Guide to Background Tasks, Workers, and Parallel Processing in Python

Introduction to Celery and Its Uses

I Just Discovered Python’s super() Works Differently Than I Thought

I always thought that I knew how super() worked in Python. But one incident broke this illusion.