S3 connection error with Redshift provider

  • danielbos
  • Topic Author
  • Visitor
  • Visitor
9 years 3 months ago #10434 by danielbos
Hi all,

I have created a SSIS package with OLE DB connections which inserts data to a simple table. I wanted to test the speed of the PGNP Redshift connector.

Since PGNP also uses the COPY command I have set the S3 information like:

Bulk insert method: Copy command
Bulk insert buffer size: 1000000
AWS Access Key: myaccesskey
AWS Secret Key: mysecretkey
AWS Bucket: s3:\\mybucketname

When I test the package I'll get an error:
[OLE DB Destination [2]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Redshift OLE DB Provider" Hresult: 0x80004005 Description: "ERROR: S3ServiceException:The specified bucket does not exist,Status 404,Error NoSuchBucket

What am I doing wrong? I have set various versions of the AWS Bucket: for example just mybucketname or [url=s3://mybucketname]s3://mybucketname[/url] or s3:\\mybucketname\

Could you help me out?

Daniel

Please Log in or Create an account to join the conversation.

More
9 years 3 months ago #10435 by Moderator
Hi Daniel,
please do not add any prefix to the bucket name, just keep it as "mybucketname".

Also, I do not recommend using bulk insert size of 1000000. Depending on a row size, it could stress RAM. Start with 10000, and see if it satisfies your performance goal.

Best regards,
Moderator

Please Log in or Create an account to join the conversation.

  • danielbos
  • Topic Author
  • Visitor
  • Visitor
9 years 3 months ago #10437 by danielbos
Replied by danielbos on topic Re: S3 connection error with Redshift provider
That did the trick!

Thanks!

Please Log in or Create an account to join the conversation.

Time to create page: 0.151 seconds
Powered by Kunena Forum