Skip to content

ARROW-6001 [Python]: Add from_pylist() and to_pylist() to pyarrow.Table to convert list of records - #12010

Closed
AlenkaF wants to merge 13 commits into
apache:masterfrom
AlenkaF:ARROW-6001
Closed

AlenkaF wants to merge 13 commits into
apache:masterfrom
AlenkaF:ARROW-6001

Conversation

@AlenkaF

@AlenkaF AlenkaF commented Dec 21, 2021

Copy link
Copy Markdown
Member

Add to_pylist and from_pylist to Table and RecordBatch.

to_pylist returns a list of dicts
from_pylist returns Table/RecordBatch from a list of dicts (named mapping in the code)

@github-actions

Copy link
Copy Markdown

Comment thread python/pyarrow/table.pxi Outdated
Comment thread python/pyarrow/table.pxi Outdated
Comment thread python/pyarrow/table.pxi Outdated
Comment thread python/pyarrow/table.pxi Outdated
@AlenkaF

AlenkaF commented Jan 4, 2022

Copy link
Copy Markdown
Member Author

I have corrected the code so that pylist is meant to be structured as a list of dicts, one dict per row. If the column name is missing from the schema or from the data, None is put in.

If the schema is not given in from_pylist(), the keys from the first dictionary are used to define the names of the columns in the Table/RecordBatch.

Also I saw that there are only a few examples added in the docstrings. I think they are a great way to understand the codebase as they are easy to locate and to use. I would be happy to add them as a part of the Documentation Improvements or as a separate JIRA issue.

Comment thread python/pyarrow/table.pxi Outdated

@jorisvandenbossche jorisvandenbossche left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The behaviour looks good now!

Comment thread python/pyarrow/table.pxi Outdated
Comment thread python/pyarrow/table.pxi Outdated
Comment thread python/pyarrow/table.pxi Outdated
Comment thread python/pyarrow/table.pxi Outdated
Comment thread python/pyarrow/table.pxi Outdated
Comment thread python/pyarrow/table.pxi Outdated
Comment thread python/pyarrow/table.pxi Outdated
Comment thread python/pyarrow/table.pxi Outdated
@jorisvandenbossche

Copy link
Copy Markdown
Member

@AlenkaF there is also a linter error (probably due to my suggestions)

@AlenkaF

AlenkaF commented Jan 11, 2022

Copy link
Copy Markdown
Member Author

Will pull the changes and correct, thanks for the ping!

AlenkaF and others added 3 commits January 11, 2022 10:51
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
@jorisvandenbossche

Copy link
Copy Markdown
Member

Thanks @AlenkaF !

@AlenkaF
AlenkaF deleted the ARROW-6001 branch January 11, 2022 13:40
@ursabot

ursabot commented Jan 11, 2022 •

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 7a0141a and contender = ccffcea. ccffcea is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed] ursa-i9-9960x
[Finished ⬇️0.22% ⬆️0.0%] ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants