AWS SDK for pandas 2.17.0

New Functionalities

• RedshiftDataAPI serverless support 🔥 #1530
  • Check out the tutorial
• Add get_query_results to the Athena module #1496
  • Check out the function documentation
• Add generate_create_query to the Athena module #1514
  • Check out the function documentation

Enhancements

• Returning empty DataFrame for empty TimeStream query #1430
• Added support for INSERT IGNORE for mysql.to_sql #1429
• Added use_column_names to redshift.copy akin to redshift.to_sql #1437
• Enable passing kwargs to redshift.connect #1467
• Add timestream_endpoint_url property to the config #1483
• Add support for upserting to an empty Glue table #1579

Documentation

• Fix typos in documentation #1434

Bug Fix

• validate_schema=True for wr.s3.read_parquet breaks with partition columns and dataset=True #1426
• wr.neptune.to_property_graph failing for Neptune version 1.1.1.0 #1407
• ValueError when using opensearch.index_df with documents with an array field #1444
• Missing catalog_id in wr.catalog.create_database #1480
• Check for pair of brackets in query preparation for Athena cache #1529
• Fix wrong type hint for TagColumnOperation in quicksight.create_athena_dataset #1570
• s3.to_json compression parameters is passed twice when dataset=True #1585
• Cast Athena array, map & struct types to pandas object #1581
• In the OpenSearch module, use SSL only for HTTPS (port 443) #1603

Noteworthy

AWS Lambda Managed Layers

Since the last release, the library has been accepted as an official SDK for AWS, and rebranded as AWS SDK for pandas 🚀. The module names in Python will remain the same. One noteworthy change, however, is that the AWS Lambda Manager layer name has been renamed from AWSDataWrangler to AWSSDKPandas.

You can view the ARN value for the layers here.

PyArrow 7 Support

⚠️ For platforms without PyArrow 7 support (e.g. MWAA, EMR, Glue PySpark Job):

pip install pyarrow==2 awswrangler

Thanks

We thank the following contributors/users for their work on this release:

@bechbd, @maxispeicher, @timgates42, @aeeladawy, @KhueNgocDang, @szemek, @malachi-constant, @cnfait, @jaidisido, @LeonLuttenberger, @kukushking