Let's suppose you want to create a task that loads a Pipe Delimited File into a database on a daily basis. The task is to run on Mac OS and you are using Snowflake for the database. You can do it with SDTK as follows:
1) Download SDTK. For this tutorial, we're going to use the latest version of the Python version as of this writing (0.1.2) which is here: https://sourceforge.net/projects/simple-data-toolkit/files/0.1.3/stc.py/download
2) Create a new text file called ConvertPSV.sh
3) Open the file with a text editor (like Text Edit)
Enter the following text:
#!/bin/sh
python stc.py clients.psv clients.sql createorreplace clients
(For more info on STC and SDTK see the following URL: https://www.vis-software.com/#sdtk)
export SNOWSQL_PWD=password
snowsql -a myorganization-myaccount -u jsmith -f clients.sql -d database -s public -o quiet=true -o friendly=false
(For more info on snowsql see the following URL: https://docs.snowflake.com/en/user-guide/snowsql-use.html)
4) Go to the Terminal.
5) Edit the list of cron jobs with the following command:
nano crontab -e
6) Add the following to a new line:
0 8 * * * cd ~ && ConvertPSV.sh
7) Press CTRL+O and CTRL+X to save and exit nano.
from sdtk import com_sdtk_api_ChatGPTAPI import os import fnmatch import re texData = "" texBegin = "\n\\begin{document}\n" texEnd = "\n\\end{document}" header = "" footer = "" foundBody = False # Get a list of all .png files in the current directory png_files = [file for file in os.listdir('.') if fnmatch.fnmatch(file, '*.png')] def result(text): global texData global header global footer global foundBody text = text.replace("```latex", "").replace("```", "") body_match = re.search(r'\\begin{document}(.*?)\\end{document}', text, re.DOTALL) if body_match: body_content = body_match.group(1).strip() if len(texData) > 0: texData = texData + "\n" + "\\newpage" + "\n" else: header_match = re.search(r'^(.*?)\\begin{document}', text, re.DOTALL) header = header_match.group(1).strip() footer_match = re.search(r'\\end{document}(.*)$', text, re.DOTALL) footer = footer_match.group(1).strip() texData = texData + body_content foundBody = True else: foundBody = False print(text) # Loop over each .png file for file_name in png_files: foundBody = False while foundBody == False: com_sdtk_api_ChatGPTAPI.instance().query(result, "Can you generate a LaTeX document to represent the text in this image and format it correctly and return only the LaTeX code?", file_name) with open("output.tex", "w+") as file: # Write the string to the file file.write(header + texBegin + texData + texEnd + footer)Atotium News - Business Aggregate - 2024-12-10Federal Reserve to cut rates by 25 bps on Dec. 18, pause in January- Reuters poll https://finance.yahoo.com/news/federal-cut-rates-25-bps-135054884.html The U.S. Federal Reserve is expected to cut interest rates by 25 basis points on December 18, with 90% of economists polled by Reuters predicting this move. Following this, a pause in rate changes is anticipated in late January amidst concerns over rising inflation risks. Economists attribute potential inflationary pressures to President-elect Donald Trump's proposed policies, including import tariffs and tax cuts. The job market's resilience has reinforced expectations for a rate cut, with most economists forecasting the federal funds rate to be reduced to 4.25%-4.50%. However, a majority expect the Fed to hold rates steady at its January meeting. While nearly 60% of economists predict at least three more rate cuts next year, there is no consensus on future actions. The Fed aims to reach a neutral rate that neither stimulates nor restricts the economy, recently assessed at around 2.9%. Inflation forecasts have been elevated, with concerns about inflation risks due to increased tariffs and potential trade disruptions from the incoming administration. The Fed will update its economic forecasts at the December meeting. GM exits robotaxi market, will bring Cruise operations in house https://www.cnbc.com/2024/12/10/gm-halts-funding-of-robotaxi-development-by-cruise.html General Motors (GM) has decided to end its funding for the development of robotaxis by its subsidiary Cruise. Instead, GM plans to integrate Cruise into its larger technical team to focus on developing autonomous systems for personal vehicles. This decision, announced after an investment of over $10 billion in Cruise, is attributed to the competitive robotaxi market, capital allocation priorities, and the extensive time and resources required to scale up. GM bought Cruise in 2016 and plans to increase its ownership to over 97% by early 2025. This move will also reduce Cruise's annual expenditure from about $2 billion by more than half. Cruise, which halted its driverless operations in October 2023, faced regulatory issues including a crash involving a pedestrian and internal problems that contributed to the decision. GM's departure from the robotaxi market occurs as competitors like Waymo, Tesla, and others advance their own autonomous vehicle services. Microsoft Unveils Zero-Water Data Centers to Reduce AI Climate Impact https://finance.yahoo.com/news/microsoft-unveils-zero-water-data-170002064.html Microsoft has introduced a new data center design that requires zero water for cooling its chips and servers, aimed at reducing the climate impact of its expanding data center operations. This new system employs a "closed loop" method, recycling water that is initially introduced during construction without needing fresh supplies afterward. This initiative, launched in August, is expected to save over 125 million liters of water per year per data center. While the data centers will still need fresh water for facilities like bathrooms and kitchens, this new design is part of Microsoft s effort to address the environmental demands of its growing AI services, particularly in hot and dry regions such as Arizona and Texas. Existing data centers will maintain older technologies, but new facilities in Phoenix and Mount Pleasant, Wisconsin, will adopt this zero-water design starting in 2026. Crypto Trading Frenzy Leads Analyst to Hike Price Targets for Coinbase, Robinhood Stock https://www.barrons.com/articles/crypto-coinbase-robinhood-stock-price-target-71f031b2 The article discusses a recent surge in crypto trading activity, prompting an analyst from Needham & Co., John Todaro, to raise price targets for Coinbase and Robinhood. Despite previous muted trading volumes, both platforms are experiencing increased activity. Todaro increased his target for Coinbase to $420 and for Robinhood to $52, while maintaining a Buy rating. The appointment of Donald Trump and his regulatory policy changes favoring the crypto industry are seen as beneficial for these companies. SEC leadership changes and Trump's pro-crypto stance are expected to promote new products and trading on these platforms, with Robinhood aiming to be a leader in crypto and options trading by 2027 and equities by 2029. Bitcoin has reached record highs, motivating a resurgence in trading activity after a period of relative quiet.Simple Data Toolkit - Tutorial - Git API - PythonSimple Data Toolkit provides an unofficial API for reading files, commits, branches, and repos from the Git API. (At the time of this writing, the release of this is pending for complete support, but it is coming soon) To retrieve all repos a user has using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_GitAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_GitAPI.reposAPI().retrieveData({"owner": "Vis-LLC"}, printer)To retrieve all branches a repo has using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_GitAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_GitAPI.branchesAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit"}, printer)To retrieve all the files in a branch using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_GitAPI def printer(data, reader): print(reader.toArrayOfNativeMaps(None)) com_sdtk_api_GitAPI.filesAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit", "branch": "main"}, printer)To retrieve the data in a file using Simple Data Toolkit, we can do the following:
from sdtk import com_sdtk_api_GitAPI def printerData(data, reader): print(data) com_sdtk_api_GitAPI.retrieveAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit-UI", "branch": "main", "path": "index.html"}, printerData)We can also login using a personal access token (https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)
from sdtk import com_sdtk_api_GitAPI def printerData(data, reader): print(data) com_sdtk_api_GitAPI.instance().setKey("Personal Access Token Here").retrieveAPI().retrieveData({"owner": "Vis-LLC", "repo": "Simple-Data-Toolkit-UI", "branch": "main", "path": "index.html"}, printerData)
Write the right way.
Ortingo is a platform that makes publishing easier and information more accessible. Publish from a wide spectrum of various topics and connect with your audience with new ways of writing articles. Be part of a wealth of new information, through Ortingo.
Ready to write the right way?
Learn more about Ortingo
Any thoughts on Franklin's post?
To comment or reply, you need an Ortingo account.
Sign in or sign upHere's what Ortingoers think of Franklin's post.
There are no comments on this post.