Let's suppose you want to create a task that loads a Pipe Delimited File into a database on a daily basis. The task is to run on Windows and you are using Snowflake for the database. You can do it with SDTK as follows:
1) Download SDTK. For this tutorial, we're going to use the latest version of the Windows Shell Script version as of this writing (0.1.2) which is here: https://sourceforge.net/projects/simple-data-toolkit/files/0.1.3/stc-wsh.js/download
2) Create a new text file called ConvertPSV.cmd
3) Open the file with a text editor (like Notepad or Visual Studio Code)
Enter the following text:
cscript stc-wsh.js clients.psv clients.sql createorreplace clients
(For more info on STC and SDTK see the following URL: https://www.vis-software.com/#sdtk)
(For more info on cscript see the following URL: https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/cscript)
SET SNOWSQL_PWD=password
snowsql -a myorganization-myaccount -u jsmith -f clients.sql -d database -s public -o quiet=true -o friendly=false
(For more info on snowsql see the following URL: https://docs.snowflake.com/en/user-guide/snowsql-use.html)
4) Go to the Start Menu.
5) Search for Task Scheduler.
(For more info on Task Scheduler see the following URL: https://docs.microsoft.com/en-us/windows/win32/taskschd/about-the-task-scheduler)
6) Open Task Scheduler.
7) Click Create Task.
8) For name enter: Convert PSV to SQL
9) Click Triggers Tab
10) Click New button
11) Select Daily
12) Start start time to 8:00 AM.
13) Click the OK button.
14) Click Actions tab.
15) Click New button
16) For Program/script click Browse.
17) Find the ConvertPSV.cmd script.
18) Click Open.
19) Set Start in to the directory that the ConvertPSV.cmd script is in.
20) Click the OK button.
21) Click the OK button.
22) To test, right click Convert PSV to SQL.
23) Select Run.
from sdtk import com_sdtk_api_ChatGPTAPI import os import fnmatch import re texData = "" texBegin = "\n\\begin{document}\n" texEnd = "\n\\end{document}" header = "" footer = "" foundBody = False # Get a list of all .png files in the current directory png_files = [file for file in os.listdir('.') if fnmatch.fnmatch(file, '*.png')] def result(text): global texData global header global footer global foundBody text = text.replace("```latex", "").replace("```", "") body_match = re.search(r'\\begin{document}(.*?)\\end{document}', text, re.DOTALL) if body_match: body_content = body_match.group(1).strip() if len(texData) > 0: texData = texData + "\n" + "\\newpage" + "\n" else: header_match = re.search(r'^(.*?)\\begin{document}', text, re.DOTALL) header = header_match.group(1).strip() footer_match = re.search(r'\\end{document}(.*)$', text, re.DOTALL) footer = footer_match.group(1).strip() texData = texData + body_content foundBody = True else: foundBody = False print(text) # Loop over each .png file for file_name in png_files: foundBody = False while foundBody == False: com_sdtk_api_ChatGPTAPI.instance().query(result, "Can you generate a LaTeX document to represent the text in this image and format it correctly and return only the LaTeX code?", file_name) with open("output.tex", "w+") as file: # Write the string to the file file.write(header + texBegin + texData + texEnd + footer)Atotium News - Business Aggregate - 2024-11-17Title: Bosses Are Calling Workers Back to the Office. That’s Good News for Landlords. URL: https://www.wsj.com/real-estate/commercial/back-to-office-workers-landlords-e5e15663 Summary: Companies are increasingly requiring employees to return to the office, reversing the trend of remote work during the pandemic. As a result, the office market is showing signs of stabilization, with investor interest growing in office properties, especially those with amenities. Despite the challenges of high vacancy rates and obsolete spaces, the amount of occupied office space remains steady. Some companies, like Amazon and Dell, are mandating five-day office attendance, shifting power back to managers as the white-collar workforce growth slows. The stabilization, however, doesn't equate to a full recovery due to ongoing defaults and expected space reductions when leases mature. Yet, certain markets are thriving with expansion from financial and AI firms. Enhanced office amenities are drawing employees back, boosting investor confidence and leading to significant financing deals, notably the refinancing of Rockefeller Center by Tishman Speyer. Title: Deficit Threat Drives Bond Yields Higher URL: https://www.wsj.com/finance/investing/deficit-threat-drives-bond-yields-higher-6a043d44 Summary: The article discusses how concerns over a rising federal budget deficit are causing a significant increase in bond yields, especially following a recent lackluster auction of government bonds. Investors anticipate sustained high deficits regardless of the upcoming election outcome, though they expect the deficit to expand most under a Republican victory due to potential tax cuts. The Treasury Department is likely to maintain its high level of debt sales, with potential for further increases. As longer-term Treasury yields rise in reaction to increased bond supply fears, analysts are watching upcoming Treasury borrowing announcements for signals on future auction plans. The fiscal year 2025 deficit projection stands near $2 trillion, with election results potentially affecting future fiscal conditions. Title: Your Billy Bookcase Should Have Multiple Lives, Says IKEA Sustainability Chief URL: https://www.wsj.com/articles/your-billy-bookcase-should-have-multiple-lives-says-ikea-sustainability-chief-f30b24e8 Summary: IKEA's sustainability efforts, led by Ingka Group's Chief Sustainability Officer Karen Pflug, focus on achieving net-zero emissions by 2050 without carbon offsets. A key initiative is the testing of Ikea Preowned, a platform in Madrid and Oslo for buying and selling secondhand IKEA furniture, which they hope to expand globally. The company aims to use renewable or recycled materials across its products while promoting circularity, encouraging customers to extend product lifecycles. Efforts include the provision of spare parts and a peer-to-peer marketplace, reflecting IKEA's commitment to sustainability without compromising product quality. Title: Used EVs Sell for Bargain Prices Now, Putting Owners and Dealers in a Bind URL: https://www.wsj.com/business/autos/used-evs-sell-for-bargain-prices-now-putting-owners-and-dealers-in-a-bind-a44e1718 Summary: The value of used electric vehicles (EVs) has dropped significantly, contrasting with the stable broader used-car market. This decline follows price cuts on new EVs, particularly led by Tesla, aimed at boosting sales but negatively impacting used EV values. Previously, in-demand models like the Tesla Model 3 and Model Y have seen a 25% price drop. Factors contributing to this dip include the influx of off-loaded Teslas from rental company Hertz and competitive lease deals on new models. The plummeting prices present challenges for current owners who find themselves owing more than their cars are worth and potential long-term concerns for the industry as leasing rises sharply. Title: Retail giant buys properties near DIA for $91M URL: https://www.bizjournals.com/denver/news/2024/09/30/amazon-aurora-porteos-dia-ambrose-property-group.html Summary: Amazon has acquired a warehouse and additional land in Aurora, Colorado, near Denver International Airport, for over $91 million. The purchases include a 625,000-square-foot building within a larger industrial development known as DIA Logistics Park, and a separate parcel of vacant land. These acquisitions expand Amazon's footprint in the region, benefitting from a location that already hosts several of the company's facilities. Title: The US retirement system gets a C+ in global study URL: https://finance.yahoo.com/news/the-us-retirement-system-gets-a-c-in-global-study-230119976.html Summary: The US retirement system received a C+ in the 16th annual Mercer CFA Institute Global Pension Index, placing 29th out of 48 countries. The evaluation highlighted concerns regarding pension funding and insufficient private retirement savings, exacerbated by declining birth rates and increasing life expectancy. The US scored a C+ for adequacy and Cs for sustainability and integrity. The report suggests incorporating automatic enrollment and contribution escalation to improve the system, taking cues from countries like the Netherlands, which have successful retirement systems. The Secure 2.0 Act, which mandates automatic enrollment and escalation for new retirement plans starting in 2025, could help address some of these issues.Simple Data Toolkit - Tutorial - Git API - PythonSimple Data Toolkit provides an unofficial API for reading files, commits, branches, and repos from the Git API. (At the time of this writing, the release of this is pending for complete support, but it is coming soon) To retrieve all repos a user has using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_GitAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_GitAPI.reposAPI().retrieveData({"owner": "Vis-LLC"}, printer)To retrieve all branches a repo has using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_GitAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_GitAPI.branchesAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit"}, printer)To retrieve all the files in a branch using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_GitAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_GitAPI.filesAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit", "branch": "main"}, printer)To retrieve the data in a file using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_GitAPI def printerData(data, reader): print(data) com_sdtk_api_GitAPI.retrieveAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit-UI", "branch": "main", "path": "index.html"}, printerData)We can also login using a personal access token (https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)
from sdtk import com_sdtk_api_GitAPI def printerData(data, reader): print(data) com_sdtk_api_GitAPI.instance().setKey("Personal Access Token Here").retrieveAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit-UI", "branch": "main", "path": "index.html"}, printerData)
Write the right way.
Ortingo is a platform that makes publishing easier and information more accessible. Publish from a wide spectrum of various topics and connect with your audience with new ways of writing articles. Be part of a wealth of new information, through Ortingo.
Ready to write the right way?
Learn more about Ortingo
Any thoughts on Franklin's post?
To comment or reply, you need an Ortingo account.
Sign in or sign upHere's what Ortingoers think of Franklin's post.
There are no comments on this post.