Skip to content

Commit 6ef778a

Browse files
docs: add screenshots to step-by-step guide in README
1 parent 193e4c7 commit 6ef778a

13 files changed

Lines changed: 66 additions & 17 deletions

managed-connectivity/community-contributed-connectors/snowflake-horizon-connector/README.md

Lines changed: 66 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ This document will show an end to end POC of how to use Snowflake to Dataplex Un
1919
#### Step 1: Setting up Snowflake Environment from where you have to load the metadata.
2020
To access the Horizon catalog in Snowflake, you will need to use the **ACCOUNT_USAGE** views located under the **SNOWFLAKE** database.
2121

22-
![Snowflake Environment Setup](images/SnowflakeSetup.png)
22+
<img alt="Snowflake Environment Setup" src="images/SnowflakeSetup.png" width="600">
2323

2424
#### Step 2: Storing the connection details in Secret Manager.
2525

@@ -36,34 +36,83 @@ Please note that you have to create these secrets in your GCP Project.
3636
You will perform the following one-time setup steps in your GCP Project:
3737

3838
1. **Create an entry group** for the entries that you want to import. Follow the steps from [here](https://docs.cloud.google.com/dataplex/docs/ingest-custom-sources#create-entry-group) and create an entry group named **"snowflakehorizongrp"**
39-
![Snowflake Group Creation](images/SnowGroupCreation.png)
39+
<img alt="Snowflake Group Creation" src="images/SnowGroupCreation.png" width="600">
4040

4141
2. **Create aspect types** for the aspects that you want to import. Follow the steps from [here](https://docs.cloud.google.com/dataplex/docs/enrich-entries-metadata#create-aspect-type) and create the following aspects types: **"snowhorizondb"**, **"snowhorizonschema"**, **"snowhorizontable"**, **"snowhorizontag"**, **"snowhorizontagref"**, **"snowhorizonindex"**, **"snowhorizonfunction"**. Please note that these names are not mandatory and can be changed as per your requirements as well just that you will have to change these in the script also. If you want to use the script as it is then please use the names as mentioned above.
4242
* Navigate to Dataplex -> Dataplex Universal Catalog -> Catalog -> "Aspect types & Tag Templates "
43-
![Create Aspect Type Screen 1](images/CreateAspectTypeScreen1.png)
43+
<img alt="Create Aspect Type Screen 1" src="images/CreateAspectTypeScreen1.png" width="600">
44+
4445
* You will see two options there as shown above:- "Custom" & "System". Under "Custom", click on "+Create Aspect Type". You will see the following screen:-
45-
![Create Aspect Type](images/CreateAspectTypeSubstep.png)
46+
<img alt="Create Aspect Type" src="images/CreateAspectTypeSubstep.png" width="600">
47+
4648
* Fill the details as below(e.g for snowhorizondb)
47-
![Fill in Details](images/FillinDetailsAspect.png)
49+
<img alt="Fill in Details" src="images/FillinDetailsAspect.png" width="600">
50+
4851
* Next, Under "Template", click on "Add Field":-
49-
![Add Field](images/AddField.png)
52+
<img alt="Add Field" src="images/AddField.png" width="600">
53+
5054
* Fill all the details related to that field(e.g DatabaseName):-
51-
![New Field](images/NewFieldScreen.png)
52-
* Similarly, add all the fields(metadata that you are trying to bring from Snowflake Horizon) and click on "Save":-
53-
![FinalAspectType](images/FinalAspectType.png)
54-
Similarly, create all the aspect types mentioned above one by one. Required Fields are mentioned in the python script for each aspect type(if you plan to execute the script as it is).
55-
![Aspect Type Creation](images/AspectTypeCreation.png)
55+
<img alt="New Field" src="images/NewFieldScreen.png" width="600">
56+
57+
* Similarly, add all the fields(metadata that you are trying to bring from Snowflake Horizon):-
58+
<img alt="FinalAspectType" src="images/FinalAspectType.png" width="600">
59+
60+
* Similarly, create all the aspect types mentioned above as shown in the below screenshots. Please note that these are sample templates(just for reference purpose only) and you can modify these as per your business usecase. If you plan to execute the python script as it is then please create the Aspect Types as per the below screenshots.
61+
62+
* Aspect Type Template For Snowflake Schema:-
63+
<img alt="Snowflake Horizon Schema" src="images/AspectType_SnowHorizonschema.png" width="600">
64+
65+
* Aspect Type Template For Snowflake Index:-
66+
<img alt="Snowflake Horizon Index" src="images/AspectType_SnowHorizonIndex.png" width="600">
67+
68+
* Aspect Type Template For Snowflake Function:-
69+
<img alt="Snowflake Horizon Function" src="images/AspectType_SnowHorizonFunction.png" width="600">
70+
71+
* Aspect Type Template For Snowflake Table:-
72+
<img alt="Snowflake Horizon Table" src="images/AspectType_SnowHorizonTable.png" width="600">
73+
74+
* Aspect Type Template For Snowflake Tag:-
75+
<img alt="Snowflake Horizon Tag" src="images/AspectType_SnowHorizonTag.png" width="600">
76+
77+
* Aspect Type Template For Snowflake Tag References:-
78+
<img alt="Snowflake Horizon Tag" src="images/AspectType_SnowHorizonTagRef.png" width="600">
79+
80+
* Finally, you will have all the aspect types created as below:-
81+
<img alt="Aspect Type Creation" src="images/AspectTypeCreation.png" width="600">
5682

5783
3. **Create entry types** for the entries that you want to import. Follow the steps from [here](https://docs.cloud.google.com/dataplex/docs/ingest-custom-sources#create-entry-type) and create the following entry types: **"snowhorizondb"**, **"snowhorizonschema"**, **"snowhorizontable"**, **"snowhorizontag"**, **"snowhorizontagref"**, **"snowhorizonindex"**, **"snowhorizonfunction"**. Please note that these names are not mandatory and can be changed as per your requirements as well just that you will have to change these in the script also. If you want to use the script as it is then please use the names as mentioned above.
5884
* Navigate to Dataplex -> Dataplex Universal Catalog -> Catalog -> "Aspect types & Tag Templates "
59-
![Entry Type Creation 1](images/EntryTypeScreen1.png)
85+
<img alt="Entry Type Creation 1" src="images/EntryTypeScreen1.png" width="600">
86+
6087
* You will see two options there as shown above:- "Custom" & "System". Under "Custom", click on "+ Create". You will see the following screen, Fill in the details as below(e.g for snowhorizondb):-
61-
![Entry Type Creation 2](images/EntryTypeScreen2.png)
88+
<img alt="Entry Type Creation 2" src="images/EntryTypeScreen2.png" width="600">
89+
6290
* Under, "Required Aspect Type", click on "Choose aspect type" and select the relevant aspect type as created in Step 2 and click on "Save":-
63-
![Entry Type Creation 3](images/EntryTypeScreen3.png)
64-
![Entry Type Creation 4](images/EntryTypeScreen4.png)
65-
Similarly, create all the above mentioned Entry Types and attach the relevant Aspect Type (created in step 2) to it.
66-
![Entry Type Creation](images/EntryTypeCreation.png)
91+
<img alt="Entry Type Creation 3" src="images/EntryTypeScreen3.png" width="600">
92+
<img alt="Entry Type Creation 4" src="images/EntryTypeScreen4.png" width="600">
93+
94+
* Similarly, create all the above mentioned Entry Types and attach the relevant Aspect Type to it(as shown in the below screenshots). Please note that these are sample Entry Types(just for reference purpose only) and you can modify these as per your business usecase. If you plan to execute the python script as it is then please create the Entry Types as per the below screenshots.
95+
96+
* Entry Type For Snowflake Schema:-
97+
<img alt="Snowflake Horizon Schema" src="images/EntryType_SnowHorizonSchema.png" width="600">
98+
99+
* Entry Type For Snowflake Index:-
100+
<img alt="Snowflake Horizon Index" src="images/EntryType_SnowHorizonIndex.png" width="600">
101+
102+
* Entry Type For Snowflake Function:-
103+
<img alt="Snowflake Horizon Function" src="images/EntryType_SnowHorizonFunction.png" width="600">
104+
105+
* Entry Type For Snowflake Table:-
106+
<img alt="Snowflake Horizon Table" src="images/EntryType_SnowHorizonTable.png" width="600">
107+
108+
* Entry Type For Snowflake Tag:-
109+
<img alt="Snowflake Horizon Tag" src="images/EntryType_SnowHorizonTag.png" width="600">
110+
111+
* Entry Type For Snowflake Tag References:-
112+
<img alt="Snowflake Horizon Tag References" src="images/EntryType_SnowHorizonTagRef.png" width="600">
113+
114+
* Finally, you will have all the Entry Types created as below:-
115+
<img alt="Entry Type Creation" src="images/EntryTypeCreation.png" width="600">
67116

68117
#### Step 4: Execute the Python Script to load the Horizon Metadata from Snowflake to Dataplex
69118
Now you have to execute the Python script: `snowflake_to_dataplex_metadata_loader.py`
135 KB
Loading
152 KB
Loading
131 KB
Loading
137 KB
Loading
138 KB
Loading
146 KB
Loading
166 KB
Loading
166 KB
Loading
167 KB
Loading

0 commit comments

Comments
 (0)