emr

In order to get information from an existing EMR cluster, we can use 1 PS S:\ Get-EMRCluster -ClusterId $ClusterId The command will then return a system object in Amazon.ElasticMapReduce.Model.Cluster type. The Cluster object provides the following attributes that maybe useful MasterPublicDnsName. The DNS name of the master node. NormalizedInstanceHours. An approximation of the cost of the cluster. ReleaseLabel. The release label of Amazon EMR. Status. The current status details about the cluster....

I came across this error this morning with EMR and Spark steps. 1 An error occurred (ValidationException) when calling the RunJobFlow operation: 1 validation error detected: Value '[ <YOUR-SPARK-JOB> ]' at 'steps.45.member.hadoopJarStep.args' failed to satisfy constraint: Member must satisfy constraint: [Member must have length less than or equal to 10280, Member must have length greater than or equal to 0, Member must satisfy regular expression pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*] Or 1 botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the RunJobFlow operation: Size of step parameter length exceeded the maximum allowed....

Reference AWS Big Data Blog: Best practices for resizing and automatic scaling in Amazon EMR