-
Notifications
You must be signed in to change notification settings - Fork 286
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Raised ArrowTypeError
should indicates which column is causing the error
#1822
Comments
Thank you @aaaaahaaaaa for raising the issue. This error message seems to be raised directly by pyarrow, and we might be able to wrap some more useful information outside of it. Could you provide a minimal code snippet that reproduces the issue, so I can locate the exact place the exception is raised? |
Here is a minimal example: import google.auth
import pandas as pd
from google.cloud import bigquery
credentials, project = google.auth.default()
client = bigquery.Client(credentials=credentials, project=project)
table_id = "TABLE_ID"
data = {"what": "ever"}
df = pd.DataFrame([data])
job_config = bigquery.LoadJobConfig(schema=[bigquery.SchemaField("what", "INTEGER")])
job = client.load_table_from_dataframe(df, table_id, job_config=job_config)
job.result() Error (doesn't show any debugging information referring to the field
|
At this time, I am pushing to get PR #1836 finished so we can close this out. It appears that we have about three PRs generated by three different folks that are related and focused on similar problems. Aiming to have #1836 be the main focal point and will close out the other two PRs. Unique characteristics of each, where reasonable, will be incorporated into #1836. |
Debugging typing errors when working with existing tables with a large number of columns is extremely frustrating, to say the least: the
pyarrow
error never indicates which column is causing the issue.It would be an absolute life saver if more debugging information were raised with the exception.
E.g.:
The text was updated successfully, but these errors were encountered: