pyspark.sql.functions.explode One of the important operations in PySpark is the explode function, which is used…
Month: January 2022
How to convert Array elements to Rows in PySpark ? PySpark – Explode Example code.
Function : pyspark.sql.functions.explode To converts the Array of Array Columns to row in PySpark we use “explode” function. Explode returns…
How to find array contains a given value or values using PySpark ( PySpark search in array)
array_contains You can find specific value/values in an array using spark sql function array_contains. array_contains(array, value) will return true if…
How to removes duplicate values from array in PySpark
This blog will show you , how to remove the duplicates in an column with array elements. Consider the below example….
How to extract Date, Time into two separate cell from Timestamp in Excel ?
Consider you have a date timestamp value in cell A1 A1 = 01/21/2022 11:35 PM You want to extract date…
What are the Python libraries provided by AWS Glue Version 2.0
The defaults Python libraries available in AWS Glue version 2.0 are as below boto3==1.12.4 botocore==1.15.4 certifi==2019.11.28 chardet==3.0.4 cycler==0.10.0 Cython==0.29.15 docutils==0.15.2…
How to add additional Python Libraries in a AWS Glue Development Endpoint
There are multiple scenario that you may need to use different set of python libraries in your python code or…