Amazon Web Services

Reference TheGeekStuff: 28 Essential AWS S3 CLI Command Examples to Manage Buckets and Objects

This morning I was about to kickoff some typical jobs with AWS CLI on an Linux EC2 instance. The AWS CLI command always raised the error, 1 2 3 [user@host ~]$ aws s3 ls An error occurred (RequestTimeTooSkewed) when calling the ListBuckets operation: The difference between the request time and the current time is too large. even with the simplest list object command the error were still there!!! By querying Stack Overflow, some people had the same problem with S3 upload, and suggested to compare the local time with Amazon server-side time....

In order to get information from an existing EMR cluster, we can use 1 PS S:\ Get-EMRCluster -ClusterId $ClusterId The command will then return a system object in Amazon.ElasticMapReduce.Model.Cluster type. The Cluster object provides the following attributes that maybe useful MasterPublicDnsName. The DNS name of the master node. NormalizedInstanceHours. An approximation of the cost of the cluster. ReleaseLabel. The release label of Amazon EMR. Status. The current status details about the cluster....

I came across this error this morning with EMR and Spark steps. 1 An error occurred (ValidationException) when calling the RunJobFlow operation: 1 validation error detected: Value '[ <YOUR-SPARK-JOB> ]' at 'steps.45.member.hadoopJarStep.args' failed to satisfy constraint: Member must satisfy constraint: [Member must have length less than or equal to 10280, Member must have length greater than or equal to 0, Member must satisfy regular expression pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*] Or 1 botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the RunJobFlow operation: Size of step parameter length exceeded the maximum allowed....

According to the AWS Document , Redshift provides PG_TABLE_DEF table that contains all the table information, and you can query that table like 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 SELECT DISTINCT schemaname, tablename, column, type, encoding, distkey, sortkey, notnull FROM PG_TABLE_DEF WHERE 1=1 AND schemaname in ('<SCHEMA_NAME>') ORDER BY schemaname ASC, tablename ASC ; However, this only returns all the information in the public schema, I assume there maybe some permission problem with my account....