This repository was archived by the owner on Mar 6, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Mar 6, 2026. It is now read-only.
BigQuery: complex repeated schema fields are slow #6
Description
When converting rows to JSON, repeated schema fields are deepcopied causing noticeable slowdowns in complex schemas.
Since conversion is performed recursively, complex schemas containing multiple levels of repeated fields (e.g. repeated list of structs each containing multiple fields of repeated structs) are copied multiple times when converting a single row - causing even further slowdown due to redundant copying.
I've created a proposal for removing the deepcopy operation. Will reference relevant PR.
Activity
grooveygr commentedon Jan 21, 2020
Just to demonstrate the issue, these are the sort of profiling results one may expect when inserting a row into a complex-schema table:
