If you’re new to Unstructured, read this note first.Before you can create a destination connector, you must first sign up for Unstructured and get your
Unstructured API key. After you sign up, the Unstructured user interface (UI) appears, which you use to get the key.
To learn how, watch this 40-second how-to video.After you create the destination connector, add it along with a
source connector to a workflow.
Then run the worklow as a job. To learn how, try out the
hands-on Workflow Endpoint quickstart,
go directly to the quickstart notebook,
or watch the two 4-minute video tutorials for the Unstructured Python SDK.You can also create destination connectors with the Unstructured user interface (UI).
Learn how.If you need help, reach out to the community on Slack, or
contact us directly.You are now ready to start creating a destination connector! Keep reading to learn how.
-
A Snowflake account and its account identifier.
To get the identifier for the current Snowflake account:
- Log in to Snowsight with your Snowflake account.
- In Snowsight, on the navigation menu, click your username, and then click Account > View account details.
- On the Account tab, note the value of the Account Identifier field.
-
The Snowflake user’s login name (not its username) and its password in the account.
To view the login name for a user:
- Log in to Snowsight with your Snowflake account.
- In Snowsight, on the navigation menu, click Admin > Users & Roles.
- On the Users tab, in the list of available users, click the name of the target user.
- In the About tile, note the Login Name for the user.
<my-user>
, including theirlogin_name
value representing their login name: -
The name of the Snowflake role that the user belongs to and that also has sufficient access to the Snowflake database, schema, table, and host.
- To create a database in Snowflake, the role needs to be granted
CREATE DATABASE
privilege at the current account level; andUSAGE
privilege on the warehouse that is used to create the database. - To create a schema in a database in Snowflake, the role needs to be granted
USAGE
privilege on the database and the warehouse that is used to create the schema; andCREATE SCHEMA
on the database. - To create a table in a schema in Snowflake, the role needs to be granted
USAGE
privilege on the database and schema and the warehouse that is used to create the table; andCREATE TABLE
on the schema. - To write to a table in Snowflake, the role needs to be granted
USAGE
privilege on the database and schema and the warehouse that is used to write to the table; andINSERT
on the table. - To read from a table in Snowflake, the role needs to be granted
USAGE
privilege on the database and schema and the warehouse that is used to write to the table; andSELECT
on the table.
- Log in to Snowsight with your Snowflake account.
- In Snowsight, on the navigation menu, click Admin > Users & Roles.
- Click the Roles tab.
Grant privileges to a role. Learn more. - To create a database in Snowflake, the role needs to be granted
-
The Snowflake warehouse’s hostname and its port number in the account.
To view a list of available warehouses in the current Snowflake account:
- Log in to Snowsight with your Snowflake account.
- In Snowsight, on the navigation menu, click Admin > Warehouses. This view does not provide access to the warehouses’ hostnames or port numbers. To get this information, you must run a Snowflake query.
type
ofSNOWFLAKE_DEPLOYMENT
: -
The name of the Snowflake database in the account.
To view a list of available databases in the current Snowflake account:
- Log in to Snowsight with your Snowflake account.
- In Snowsight, on the navigation menu, click Data > Databases.
-
The name of the schema in the database.
To view a list of available schemas for a database in the current Snowflake account:
- Log in to Snowsight with your Snowflake account.
- In Snowsight, on the navigation menu, click Data > Databases.
- Expand the name of the target database.
The following Snowflake query returns a list of available schemas for the database named<database-name>
in the current account: -
The name of the table in the schema.
To view a list of available tables for a schema in a database in the current Snowflake account:
- Log in to Snowsight with your Snowflake account.
- In Snowsight, on the navigation menu, click Data > Databases.
- Expand the name of the database that contains the target schema.
- Expand the name of the target schema.
- Expand Tables.
<schema_name>
in the datbase named<database-name>
in the current account:Snowflake requires the target table to have a defined schema before Unstructured can write to the table. The recommended table schema for Unstructured is as follows:SQL -
The name of the column in the table that uniquely identifies each record (for example,
RECORD_ID
).
<name>
(required) - A unique name for this connector.<account>
(required): The target Snowflake account’s identifier.<role>
(required): The name of the Snowflake role that the user belongs to. This role must have the appropriate access to the target Snowflake warehouse, database, schema, and table.<user>
(required): The target Snowflake user’s login name (not their username).<password>
(required): The user’s password.<host>
(required): The hostname of the target Snowflake warehouse.<port>
(required): The warehouse’s port number. The default is443
if not otherwise specified.<database>
(required): The name of the target Snowflake database.<schema>
(required): The name of the target Snowflake schema within the database.<table_name>
: The name of the target Snowflake table within the database’s schema. For the destination connector, the default iselements
if not otherwise specified.<columns>
(source connector only): A comma-separated list of columns to fetch from the table. By default, all columns are fetched unless otherwise specified.<id-column>
(required, source connector only): The name of the column that uniquely identifies each record in the table.<record-id-key>
(destination connector only): The name of the column that uniquely identifies each record in the table. The default isrecord_id
if not otherwise specified.<batch-size>
(required): The maximum number of rows to fetch for each batch. The default is50
if not otherwise specified.