Repository navigation
Skip data validation when fetching data from the DB #858
Description
Problem Desctription
I have noticed that it takes a long time to return the result of simple query. At first glance, I thought it is because of some unnecessary joins. But it turns out that the queries are too fast (less than 10ms), while creating and validating pydantic models take a lot of time (~600ms).
Proposed solution
When I check pydantic docs I find this:
pydantic also provides the construct() method which allows models to be created without validation this can be useful when data has already been validated or comes from a trusted source and you want to create a model as efficiently as possible (construct() is generally around 30x faster than creating a model with full validation).
I changed the method ModelRow.from_row to use construct method:
- instance = cast("Model", cls(**item))
+ instance = cast("Model", cls.construct(**item))this significantly reduces response time. I think it is safe to skip data validation since it has been validated at creation time.
If that sounds reasonable I can create a PR to fix this.
Activity
collerek commentedon Oct 2, 2022
That's not so simple, unfortunately. A lot of things happen in
__init__of ormar models, among them registration and population of relations. Probably there is a chance to skip fields validation on loads (it already happens for pk_only models) but you cannot just switch toconstruct.Abdeldjalil-H commentedon Oct 3, 2022
@collerek
The method
constructcalls_construct_relations. So this will create the relations. There is a bug in_construct_relationsbut this can be fixed.collerek commentedon Oct 4, 2022
Ah ok, I forgot I already did inherit
constructmethod 😅Yeah we can try that (switching to construct in from_row I mean).
If you found a bug please report it in a separate issue to handle it first.
Thanks!
blazing-gig commentedon Oct 10, 2022
Hey @collerek @Abdeldjalil-H any updates on this ? Just curious to knw. Thanks.
Abdeldjalil-H commentedon Oct 10, 2022
@blazing-gig I am waiting for pr #870 to be merged. After that I'll try to work on it. There are about 22 tests that are failing.
vaibhavsagar commentedon May 28, 2024
Has there been any update on this? I'd love to be able to skip validation on load.
collerek commentedon Oct 26, 2026
Implemented in #1875: rows of models without custom validators, encrypted or base64 fields are now built directly from the database values without
__init__or re-validation (1.4–2.2× faster loading). Models that need validation keep it.