Let's suppose you want to create a task that loads a Pipe Delimited File into a database on a daily basis. The task is to run on Windows and you are using Snowflake for the database. You can do it with SDTK as follows:
1) Download SDTK. For this tutorial, we're going to use the latest version of the Windows Shell Script version as of this writing (0.1.2) which is here: https://sourceforge.net/projects/simple-data-toolkit/files/0.1.3/stc-wsh.js/download
2) Create a new text file called ConvertPSV.cmd
3) Open the file with a text editor (like Notepad or Visual Studio Code)
Enter the following text:
cscript stc-wsh.js clients.psv clients.sql createorreplace clients
(For more info on STC and SDTK see the following URL: https://www.vis-software.com/#sdtk)
(For more info on cscript see the following URL: https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/cscript)
SET SNOWSQL_PWD=password
snowsql -a myorganization-myaccount -u jsmith -f clients.sql -d database -s public -o quiet=true -o friendly=false
(For more info on snowsql see the following URL: https://docs.snowflake.com/en/user-guide/snowsql-use.html)
4) Go to the Start Menu.
5) Search for Task Scheduler.
(For more info on Task Scheduler see the following URL: https://docs.microsoft.com/en-us/windows/win32/taskschd/about-the-task-scheduler)
6) Open Task Scheduler.
7) Click Create Task.
8) For name enter: Convert PSV to SQL
9) Click Triggers Tab
10) Click New button
11) Select Daily
12) Start start time to 8:00 AM.
13) Click the OK button.
14) Click Actions tab.
15) Click New button
16) For Program/script click Browse.
17) Find the ConvertPSV.cmd script.
18) Click Open.
19) Set Start in to the directory that the ConvertPSV.cmd script is in.
20) Click the OK button.
21) Click the OK button.
22) To test, right click Convert PSV to SQL.
23) Select Run.
from sdtk import com_sdtk_api_GitAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_GitAPI.reposAPI().retrieveData({"owner": "Vis-LLC"}, printer)To retrieve all branches a repo has using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_GitAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_GitAPI.branchesAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit"}, printer)To retrieve all the files in a branch using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_GitAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_GitAPI.filesAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit", "branch": "main"}, printer)To retrieve the data in a file using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_GitAPI def printerData(data, reader): print(data) com_sdtk_api_GitAPI.retrieveAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit-UI", "branch": "main", "path": "index.html"}, printerData)We can also login using a personal access token (https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)
from sdtk import com_sdtk_api_GitAPI def printerData(data, reader): print(data) com_sdtk_api_GitAPI.instance().setKey("Personal Access Token Here").retrieveAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit-UI", "branch": "main", "path": "index.html"}, printerData)Simple Data Toolkit - Tutorial - Ortingo API - PythonAt the time of this writing, Ortingo does not have an official API. Fortunately, Simple Data Toolkit provides an unofficial API for reading posts (at the time of this writing, the release of this is pending for complete support, but it is coming soon) To retrieve all posts for a given user in Python, using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_OrtingoAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_OrtingoAPI.postsAPI().retrieveData({"owner": "60CQ59FN46SVQFXJ"}, printer)Let's suppose we want only a list of titles for a given user, we can do this instead:
from sdtk import com_sdtk_api_OrtingoAPI def printer(data, reader): print(reader.filterColumnsOnly(["title"]).toArrayOfNativeMaps(None)) com_sdtk_api_OrtingoAPI.postsAPI().retrieveData({"owner": "60CQ59FN46SVQFXJ"}, printer)We can also pull suggested content from Ortingo with the following, where the topics we are searching on are provided with the query parameter (in this case it's value is data):
from sdtk import com_sdtk_api_OrtingoAPI def printerUrls(data, reader): print(reader.filterColumnsOnly(["url"]).toArrayOfNativeMaps(None)) com_sdtk_api_OrtingoAPI.suggestionsAPI().retrieveData({"query": "data"}, printerUrls)And finally, we can also pull comments attached to a post in Ortingo with the following, where the user is myself and the post is a test post I created:
from sdtk import com_sdtk_api_OrtingoAPI def printerComments(data, reader): print(reader.filterColumnsOnly(["commentDate", "post"]).toArrayOfNativeMaps(None)) com_sdtk_api_OrtingoAPI.commentsAPI().retrieveData({"owner": "60CQ59FN46SVQFXJ", "id": "test"}, printerComments)The columns supported at the time of this writing are: - id - owner - title - subtitle - post - url For comments the following columns are supported: - id - owner - commentDate - replyTo - postSimple Data Toolkit - Tutorial - IEEE Events API - PythonSimple Data Toolkit provides an unofficial API for reading events from the IEEE API. (At the time of this writing, the release of this is pending for complete support, but it is coming soon) To retrieve all events in Python, using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_IEEEAPI,com_sdtk_calendar_IEEEEventFormat def printer(data, reader): ieee = com_sdtk_calendar_IEEEEventFormat.instance for event in reader.toArrayOfNativeMaps(None): ci = ieee.read(event) print(ci.summary) com_sdtk_api_IEEEAPI.eventsAPI().retrieveData({"limit": "2"}, printer)We can search using the following parameters: - limit - The limit to the number of events to return - start - The start datetime to search - end - The end datetime to search The columns supported at the time of this writing are: - created-at mapped to created - start-time mapped to start - end-time mapped to end - title mapped to summary - uid mapped to uidSimple Data Toolkit - Tutorial - BitTorrent API - PythonSimple Data Toolkit provides an API for retrieving transactions from BitTorrent. (At the time of this writing, the release of this is pending for complete support, but it is coming soon) To retrieve all transactions in Python, using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_BitTorrentAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_BitTorrentAPI.transactionsAPI().retrieveData({}, printer)We can also convert the transactions, to SDTK's internal event/calendar format with the BitTorrentFormat class like so:
from sdtk import com_sdtk_api_BitTorrentAPI,com_sdtk_calendar_BitTorrentFormat def printerEvents(data, reader): etherScan = com_sdtk_calendar_BitTorrentFormat.instance() for event in reader.toArrayOfNativeMaps(None): ci = etherScan.read(event) print(ci) com_sdtk_api_BitTorrentAPI.transactionsAPI().retrieveData({}, printerEvents)We can also search by address, startTimestamp, and endTimestamp.Simple Data Toolkit - Tutorial - Ethereum via Etherscan - PythonSimple Data Toolkit provides an unofficial API for reading Ethereum transactions from the Etherscan API. (At the time of this writing, the release of this is pending for complete support, but it is coming soon) To retrieve all Ethereum transactions for a particular address we can do the following:
from sdtk import com_sdtk_api_EtherscanAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_EtherscanAPI.transactionsAPI().retrieveData({"address": "0xAddress"}, printer)Replace 0xAddress with the address you wish to search on. Additionally, we can convert the transactions to our standard internal event interface like so:
from sdtk import com_sdtk_api_EtherscanAPI,com_sdtk_calendar_EtherscanFormat def printerEvents(data, reader): etherScan = com_sdtk_calendar_EtherscanFormat.instance() for event in reader.toArrayOfNativeMaps(None): ci = etherScan.read(event) print(ci.start.toString()) com_sdtk_api_EtherscanAPI.transactionsAPI().retrieveData({"address": "0xAddress"}, printerEvents)Replace 0xAddress with the address you wish to search on.
The more-in-one productivity platform
Reach your objectives faster and more efficiently with one interconnected workflow.
Want a more efficient workflow?
Learn more about Ortingo
Any thoughts on Franklin's post?
To comment or reply, you need an Ortingo account.
Sign in or sign upHere's what Ortingoers think of Franklin's post.
There are no comments on this post.