Skip to content


AWS Lambda Data Validation with Pydantic

AWS Lambda is a popular serverless computing service that allows developers to run code without provisioning or managing servers. This service is so widely used because it supports automatic scaling and offers a cost-effective pay-per-call pricing model.

AWS Lambda functions can be triggered by various AWS services and other event sources, which pass event and context data to said function. Like any other application, it's critical to structure and validate this incoming data to ensure proper execution of the function and reliability of the results.

In this article, we'll explore how Pydantic, the leading data validation library for Python, can be leveraged to structure and validate event and context data in AWS Lambda functions. We'll discuss the importance of understanding the structure of event and context data, and how Pydantic can help enhance developer experience by improving readability and maintainability of Lambda functions.

Building a product search API with GPT-4 Vision, Pydantic, and FastAPI

In previous blog posts, we showed that Pydantic is well suited to steering language models and validating their outputs.

The application of Pydantic extends beyond merely managing outputs of these text-based models. In this post, we present a guide on how to develop a product search API that uses Pydantic as a link between GPT-4 Vision and FastAPI. Pydantic will be used to structure both the data extraction processes as well as FastAPI requests and responses.

The combination of Pydantic, FastAPI, and OpenAI's GPT models creates a powerful stack for the development of AI applications, characterized by:

  • Pydantic's Schema Validation: This feature guarantees the uniformity and adherence to predefined schemas across the application, an essential factor for managing outputs from AI models.
  • FastAPI's Performance and Ease of Use: FastAPI serves as the optimal framework for crafting responsive APIs that can fulfill the requirements of AI applications. This is further enhanced by its seamless integration with Pydantic, which aids in data validation and serialization.
  • OpenAI's GPT-4 Vision Capabilities: The inclusion of GPT-4 Vision introduces a layer of advanced AI intelligence, empowering applications with the ability to accurately interpret and analyze visual data.

Minimize LLM Hallucinations with Pydantic Validators

In our previous post we introduced Pydantic as a tool to steer language models.

This post, however, shifts focus on how we can leverage Pydantic's validation mechanism to minimize hallucinations. We'll explain how validation works and explore how incorporating context into validators can enrich language model result.

The intention is by the end of this article, you'll see some examples of how we can use Pydantic to minimize hallucinations and gain more confidence in the model's output.

But before we do that, lets go over some validation basics.

Steering Large Language Models with Pydantic

In the last year, there's been a big leap in how we use advanced AI programs, especially in how we communicate with them to get specific tasks done. People are not just making chatbots; they're also using these AIs to sort information, improve their apps, and create synthetic data to train smaller task-specific models.

What is Prompt Engineering?

Prompt Engineering is a technique to direct large language models (LLMs) like ChatGPT. It doesn't change the AI itself but tweaks how we ask questions or give instructions. This method improves the AI's responses, making them more accurate and helpful. It's like finding the best way to ask something to get the answer you need. There's a detailed article about it here.

While some have resorted to threatening human life to generate structured data, we have found that Pydantic is even more effective.

In this post, we will discuss validating structured outputs from language models using Pydantic and OpenAI. We'll show you how to write reliable code. Additionally, we'll introduce a new library called instructor that simplifies this process and offers extra features to leverage validation to improve the quality of your outputs.

Pydantic V2 Is Here!

The last few months have involved a whirlwind of work, and we're finally ready to announce to official release of Pydantic V2!

Getting started with Pydantic V2

To get started with Pydantic V2, install it from PyPI:

pip install -U pydantic

Pydantic V2 is compatible with Python 3.8 and above.

See the docs for examples of Pydantic at work.

Help Us Build Our Roadmap | Pydantic

Back in February I announced Pydantic Inc., but I didn't explain what services we were building.

Today I want to provide a little more detail about what we're planning to build, and get your feedback on the components via a short survey.

In return for giving us your honest feedback, you have the option to be added to our early access list, to get invited to the closed beta of our platform once it's ready to use.

Pydantic V2 Pre Release

We're excited to announce the first alpha release of Pydantic V2!

This first Pydantic V2 alpha is no April Fool's joke — for a start we missed our April 1st target date 😢. After a year's work, we invite you to explore the improvements we've made and give us your feedback. We look forward to hearing your thoughts and working together to improve the library.

For many of you, Pydantic is already a key part of your Python toolkit and needs no introduction — we hope you'll find the improvements and additions in Pydantic V2 useful.

If you're new to Pydantic: Pydantic is an open-source Python library that provides powerful data parsing and validation — including type coercion and useful error messages when typing issues arise — and settings management capabilities. See the docs for examples of Pydantic at work.

Company Announcement | Pydantic

Update: If you find this article interesting, you might also like the TechCrunch article and Sequoia blog post about Pydantic.

I've decided to start a company based on the principles that I believe have led to Pydantic's success.

I have closed a seed investment round led by Sequoia, with participation from Partech, Irregular Expressions and some amazing angel investors including Bryan Helmig (co-founder and CTO of Zapier), Tristan Handy (founder and CEO of Dbt Labs) and David Cramer (co-founder and CTO of Sentry).

Pydantic V2 Plan

Updated late 10 Jul 2022, see pydantic#4226.

I've spoken to quite a few people about pydantic V2, and mention it in passing even more.

I owe people a proper explanation of the plan for V2:

  • What we will add
  • What we will remove
  • What we will change
  • How I'm intending to go about completing it and getting it released
  • Some idea of timeframe 😨

Here goes...