Exchanges Classes¶
These classes are primary data objects for the json returned from the exchanges.
-
class
cad_tickers.exchanges.classes.
CSESedarFilings
(categories: dict, list_items: List[dict])¶
-
class
cad_tickers.exchanges.classes.
CSETicker
(updated_at: dict, metatdata: dict, quote: dict, depth_by_order: List[dict], depth_by_price: List[dict], ticker: dict, trades: List[dict])¶
-
cad_tickers.exchanges.all_tickers.
mk_full_tickers
() → pandas.core.frame.DataFrame¶ - gets all tickers from the cse and tsx/tsxv using webmoney as a proxy
Returns: full ticker df
Canadian Securities Exchange¶
Functions to download tickers from the cse
-
cad_tickers.exchanges.cse.
add_descriptions_to_df
(df: pandas.core.frame.DataFrame, max_workers: int = 16) → pandas.core.frame.DataFrame¶ - Parameters:
- clean_df
Dataframe with with randomly selected values. Data columns are as follows:
Company Full name of the company Symbol Listing symbol from the cse exchange needs a mapper to yahoo finance Industry Enum of industry including Mining Identifier Broad category (US Cannabis) Indices Enum such as CSE Composite Currency Usually CAD Trading Date when trading started urls url to listing on cse website - max_workers
- maximum number of thread workers to have
- Returns:
- df
Dataframe descriptions in every column if valid
Company Full name of the company Symbol Listing symbol from the cse exchange needs a mapper to yahoo finance Industry Enum of industry including Mining Identifier Broad category (US Cannabis) Indices Enum such as CSE Composite Currency Usually CAD Trading Date when trading started urls url to listing on cse website description cse description scrapped from website
-
cad_tickers.exchanges.cse.
clean_cse_data
(raw_df: pandas.core.frame.DataFrame) → pandas.core.frame.DataFrame¶ Removes bad data from cse dataframe.
- Parameters:
- raw_df
Dataframe with mostly unnamed columns from pandas df import
CSE Listings Label for Company data Unnamed: 1 Listing symbol from the cse exchange needs a mapper to yahoo finance Unnamed: 2 Enum of industry including Mining Unnamed: 3 Enum such as CSE Composite Unnamed: 4 Enum such as CSE Composite Unnamed: 5 Usually CAD Unnamed: 6 empty (pandas import error, dropped) Unnamed: 7 Date when trading started
- Returns:
- clean_df
Dataframe with bad data removed
Company Full name of the company Symbol Listing symbol from the cse exchange needs a mapper to yahoo finance Industry Enum of industry including Mining Identifier Broad category (US Cannabis) Indices Enum such as CSE Composite Currency Usually CAD Trading Date when trading started urls url to listing on cse website
-
cad_tickers.exchanges.cse.
get_all_cse_tickers
(cse_df: pandas.core.frame.DataFrame) → list¶ - Parameters:
- cse_df - cleaned cse dataframe
- Returns:
- webmoney_tickers - list of webmoney cse tickers
-
cad_tickers.exchanges.cse.
get_cse_files
(filename: str = 'cse.xlsx', filetype: str = 'xlsx') → str¶ Gets excel spreadsheet from api.tsx using requests
- Parameters:
- filename: Name of the file to be saved filetype: Save as pdf or xlsx
- Returns:
- filePath returns path to file
See ://stackoverflow.com/questions/13567507/passing-csrftoken-with-python-requests
-
cad_tickers.exchanges.cse.
get_cse_tickers_df
() → pandas.core.frame.DataFrame¶ Grab cse dataframe from exported xlsx sheet
- Returns:
- clean_df
Dataframe with with randomly selected values. Data columns are as follows:
Company Full name of the company Symbol Listing symbol from the cse exchange needs a mapper to yahoo finance Industry Enum of industry including Mining Identifier Broad category (US Cannabis) Indices Enum such as CSE Composite Currency Usually CAD Trading Date when trading started urls url to listing on cse website
-
cad_tickers.exchanges.cse.
get_description_for_url
(url: str) → str¶ - Parameters:
- url - link to ticker can be empty string
- Returns:
- description - details of what the ticker does, can be empty string
Toronto Stock Exchange¶
Set of functions to scrap ticker data from the toronto stock exchange.
Will definitely split into smaller files once the graphql api becomes the main api.
-
cad_tickers.exchanges.tsx.get_ticker_data.
get_ticker_data
(symbol=<class 'str'>) → Optional[dict]¶ - Parameters:
- symbol - ticker symbol from tsx, no prefix
- Returns:
- dict - Quote By Symbol
-
class
cad_tickers.exchanges.tsx.gql_data.
GQL
¶ quote_by_symbol_query:
- graphql properties for getQuoteBySymbol are
- symbol
- name
- price
- priceChange
- percentChange
- exchangeName
- exShortName
- exchangeCode
- marketPlace
- sector
- industry
- volume
- openPrice
- dayHigh
- dayLow
- MarketCap
- MarketCapAllClasses
- peRatio
- prevClose
- dividendFrequency
- dividendYield
- dividendAmount
- dividendCurrency
- beta
- eps
- exDividendDate
- shortDescription
- longDescription
- website
- phoneNumber
- fullAddress
- employees
- shareOutStanding
- totalDebtToEquity
- totalSharesOutStanding
- sharesESCROW
- vwap
- dividendPayDate
- weeks52high
- weeks52low
- alpha
- averageVolume10D
- averageVolume30D
- averageVolume50D
- priceToBook
- priceToCashFlow
- returnOnEquity
- returnOnAssets
- day21MovingAvg
- day50MovingAvg
- day200MovingAvg
- dividend3Years
- dividend5Years
- datatype
- __typename
-
cad_tickers.exchanges.tsx.get_tickers.
get_all_tickers_data
(tickers=None, max_workers: int = 16)¶ - Parameters:
- max_workers - number of workers for ThreadPoolExecutor
- Returns:
- ticker_df - pd.DataFrame with columns of Quote By Symbol
-
cad_tickers.exchanges.tsx.get_tickers.
get_all_tsx_tickers
() → list¶ - Returns:
- all_tsx - list of tickers for toronto stock exchange
-
cad_tickers.exchanges.tsx.get_tickers.
get_tsx_tickers
(exchange='tsx') → list¶ - Parameters:
- exchange - tsx or tsxv
- Returns:
- symbol_list - list of symbols for exchange
Stock News¶
Extract news from stocks on yahoo
-
cad_tickers.news.stock_news.
find_news_link_and_text
(news_content: bs4.element.Tag) → Tuple[str, str]¶ - Finds news link from news_content.
- Assumes comments are deleted from the yahoo finance news items
- Parameters:
- news_content - html based data for the news article
- Returns:
- link_href - link in html markup link_text - link text in html markup
-
cad_tickers.news.stock_news.
find_news_source
(news_content: bs4.element.Tag) → Union[None, str]¶ Utility function to verify news format from yahoo has not changed
when grabbing data from yahoo with requests, it seems date is not returned.
- Parameters:
- news_content: html based data for the news article
- Returns:
- source - publisher of article
wrapper div around content - such as - CNW Group 2 days ago
-
cad_tickers.news.stock_news.
get_ynews_for_ticker
(ticker: str, yahoo_base_url='https://finance.yahoo.com') → List[bs4.element.Tag]¶ Returns initial news items fetched from yahoo when loading quote page. Since yahoo has lazy loading, not all items are returned. Seems like ads are not loaded because of lazy loading.
- Parameters:
- ticker - yahoo formatted ticker str yahoo_base_url - optional parameter that is the base of the request
- Returns:
- news_items - list of key html content for the news item
-
cad_tickers.news.stock_news.
scrap_news_for_ticker
(ticker: str) → List[dict]¶ Extracts webpage data from a ticker
TODO add a delay
- Parameters:
- ticker - yahoo finance ticker
- Returns:
- news_data - list of dicts extracted from webpage
- source - str
- link_href - link from post (can be relative or absolute)
- link_text - description for link
- ticker - reference to original ticker
CEO¶
To use this functionality, we require dataclasses which is natively available in python 3.7, for python 3.6 please install dataclasses from pip.
-
cad_tickers.news.ceo.main.
extract_urls
(spiels: List[dict]) → Tuple[list, str]¶ Valid List Items from ceo
-
cad_tickers.news.ceo.main.
get_new_items
(ticker: str, channel='@newswire', max_iterations=60, until=0)¶ Gets news items from ceo using ticker
- Parameters:
- ticker - stock ticker, for example APHA channel - can be @newswire, @thenewswire max_iterations - max number of requests to ceo.ca
-
cad_tickers.news.ceo.main.
get_spiels
(params: dict) → dict¶ Simple function to get ceo.ca spiels
-
class
cad_tickers.news.ceo.spiel_class.
SearchParams
(channel: str = '@newswire', filter_terms: str = 'APHA', filter_top: int = 100, load_more: str = 'top', original_scroll_height: str = 0, until: int = 1608499252326)¶ Search Params for Ceo.ca intended to get spiels.
-
channel
= '@newswire'¶
-
filter_terms
= 'APHA'¶
-
filter_top
= 100¶
-
load_more
= 'top'¶
-
original_scroll_height
= 0¶
-
until
= 1608499252326¶
-
-
cad_tickers.news.ceo.utils.
earlier_timestamp
(timestamp: int, days: int = 90) → int¶ Update timestamp and return timestamp in millseconds
-
cad_tickers.news.ceo.utils.
news_link_from_spiel
(spiel: dict) → Optional[str]¶
-
cad_tickers.news.ceo.utils.
params_to_dict
(sp: Type[cad_tickers.news.ceo.spiel_class.SearchParams]) → dict¶ utility function to get query parameters
IIROC Halts¶
Find out what latest stocks have been halted from iiroc (only canada)
-
cad_tickers.news.iiroc_halts.
get_halts_resumption
() → pandas.core.frame.DataFrame¶ Gets the latest 25 halts from the iiroc
- Returns:
- halt_df
Dataframe with bad data removed
Halts Details of halts Listing Extracted ticker from halt
Sedar Documents¶
Utilities for downloading documents from sedar. Directly downloading files from sedar is difficult, so I am piggybacking the various exchanges, for the cse you can get links directly and download files.
As for the tsx and tsxv, it is a link that redirects to another website. This is fine as I do not intend to scrap data and only read the pdfs.
-
cad_tickers.sedar.cse.
get_cse_sedar_docs
(cse_data: Union[dict, cad_tickers.exchanges.classes.CSETicker]) → Union[dict, cad_tickers.exchanges.classes.CSESedarFilings]¶ - Parameters:
- cse_data - information for a ticker when it is loaded on the thecse website
- Returns:
- filings - class or dict with properties
cad_tickers.exchanges.classes.CSESedarFilings
-
cad_tickers.sedar.cse.
get_cse_ticker_data
(ticker, get_dict: bool = True) → Union[cad_tickers.exchanges.classes.CSETicker, dict]¶ - Parameters:
- ticker - stock ticker for the cse without exchange (ex, CMC) get_dict - flag to get dict
- Returns:
- cse_ticker - dict or python class containing ticker data
-
cad_tickers.sedar.tsx.
get_ticker_filings
(symbol: str, fromDate: str = '2020-12-01', toDate: str = '2020-12-20', limit: int = 100) → Optional[dict]¶ - Parameters:
- symbol - ticker symbol from tsx, no prefix fromDate - start date to grab documents toDate - end date to grab documents limit - max number of documents to retrieve
- Returns:
- dict - Quote By Symbol
Stock Utilities¶
Contains various utility classes
-
cad_tickers.util.utils.
cse_ticker_to_webmoney
(cse_ticker: str)¶ - Parameters:
- cse_ticker - cse ticker name
- Returns:
- webmoney_ticker - ticker that can be looked up in webmoney
-
cad_tickers.util.utils.
cse_ticker_to_yahoo
(row: pandas.core.series.Series) → str¶ - Parameters:
- row - series from cse dataframe
- Returns:
- ticker - yahoo ticker for cse
-
cad_tickers.util.utils.
make_cse_path
(raw_ticker: str, raw_industry: str) → str¶ makes slug for ticker for the cse
- Parameters:
- raw_ticker - cse ticker from xlsx sheet raw_industry - verbatim industry from ticker, not slugified
- Returns:
- description - url for cse files for download
- Parameters:
- description_tags - html tags from webpage, usually p tag containing description
- Returns:
- description - description for ticker
-
cad_tickers.util.utils.
read_df_from_file
(file_path: str) → pandas.core.frame.DataFrame¶ - Parameters:
- file_path - path to data
- Returns:
- df - excel sheet dataframe
-
cad_tickers.util.utils.
tickers_to_ytickers
(tsx_path: str, cse_path: str) → List[str]¶ - Parameters:
- tsx_path - path to clean tsx file cse_path - path to clean cse file
- Returns:
- ytickers - list of tickers
-
cad_tickers.util.utils.
transform_name_to_slug
(raw_ticker: str) → str¶ - Parameters:
- raw_ticker - cse ticker to be converted to slug
- Returns:
- transformed - raw_ticker
-
cad_tickers.util.utils.
tsx_ticker_to_yahoo
(row: pandas.core.series.Series) → str¶ - Parameters:
- row - pd.Series
- Ticker - ticker from pandas dataframe from cad_tickers
- Ex. - what exchange the ticker is for
- Returns:
- yticker - yahoo finance ticker for tsx
Examples¶
Grab Descriptions for all tsx tickers
from cad_tickers.exchanges.tsx import dl_tsx_xlsx, add_descriptions_to_df_pp
from datetime import datetime
start_time = datetime.now()
df = dl_tsx_xlsx()
# df = add_descriptions_to_df(df)
df = add_descriptions_to_df_pp(df)
end_time = datetime.now()
df.to_csv('tsx_all_descriptions.csv')
print(end_time - start_time)