In this article, I will show how you can structure and explore the content of your own articles using graph technology and some programming.
The idea of using NLP techniques for structuring unstructured data is not new, however, the latest progress in LLMs (Large Language Models) has sparked countless opportunities for doing just that. The accessibility for amateurs through the booming technology Chat-GPT has created a lot of attention towards LLMs and generator models.
In fact, generative AI is on the agenda in many companies already!
The way we will work with the technology in this article is through the programming language Python using OpenAI’s developer API. We will work on data from Medium (meta huh?) and build a knowledge graph. That may sound like a mouthful, but it is actually surprisingly easy to get started with.
First things first. The plan of attack is the following.
- Get the API to work and access it through Python.
- Use a sample text to do prompt engineering ensuring that the GPT-4 model understands what you want from it.
- Download your articles from Medium (you can of course use other pieces of text if you want) and pre-process the data.
- Extract and collect output from Chat-GPT.
- Post-process the output from Chat-GPT
- Write code to structure the data further into a graph using the Cypher query language.
- Play around with your new best friend and explore your articles.
Without further ado, let’s get started by quickly setting up the basic tech.
We need to have the programming language Python and the graph database Neo4j installed on our local computer.
The first thing to do is to ensure that you have a plus account at OpenAI so that you can use GPT-4. The…