ny-gov/database-of-economic-incentives-26ei-n4eb
Loading...

Query the Data Delivery Network

Query the DDN

The easiest way to query any data on Splitgraph is via the "Data Delivery Network" (DDN). The DDN is a single endpoint that speaks the PostgreSQL wire protocol. Any Splitgraph user can connect to it at data.splitgraph.com:5432 and query any version of over 40,000 datasets that are hosted or proxied by Splitgraph.

For example, you can query the database_of_economic_incentives table in this repository, by referencing it like:

"ny-gov/database-of-economic-incentives-26ei-n4eb:latest"."database_of_economic_incentives"

or in a full query, like:

SELECT
    ":id", -- Socrata column ID
    "quarter_number_of_the_submission", -- All: State fiscal year and quarter for which the project was last submitted/updated to the database.
    "ein_of_the_recipient", -- All: Federal Tax ID number (Employer Identification number) of project recipient, if applicable.
    "lead_agency_name", -- All: Name of the lead agency.
    "lead_agency_code", -- All: Unique identifier for the lead agency.
    "project_id_number", -- All: Unique identifier for each project.
    "program_through_which_the_funding_was_awarded", -- All: Name of program or funding source that is funding the economic assistance from reporting agency. 
    "recipient_name", -- All: Name of organization receiving economic assistance from reporting agency.
    "name_of_project", -- All: Name of project.
    "is_the_award_to_a_public_or_private_sector_entity_", -- All: "Public" entity being any governmental entity, including but not limited to IDAs, cities, towns, villages, or counties. "Private" entity being any entity that does not fall under the definition of “Public” entity, including but not limited to private businesses and non-profit organizations.
    "project_description", -- DOS: A brief description of the project, derived from the program, recipient name, and project name. ESD: Derived from type of benefit awarded, industry, region and name of recipient. HCR: A short description of the project, grant amount, and work involved. NYPA: Overview of project as defined in the awardee’s application and documented in their fund benefits agreement. NYSERDA: Project name as defined by the managing NYSERDA team.
    "industry", -- All: Primary industry sector of the project, based upon the two-digit NAICS code for recipient.
    "if_the_project_is_a_member_item_the_originating_district", -- All: If the project is a Member Item, the originating district.
    "street_address", -- All: Street or building location of project; projects with multiple addresses either list a primary address or “Various”. Some exact locations are approximated.
    "county", -- All: County for location of project.
    "postal_code", -- All: Zip code for location of project.
    "economic_development_region", -- All: Region for location of project (derived from the 10 New York State Economic Development Regions).
    "start_date", -- DOS: Start date of the contract ESD: In general, “Start Date” is defined as the date on which a project is deemed to be “Approved” by ESD. For purposes of the Database, a project is “Approved” based on the most relevant milestone for each project, as required milestones and approval procedures vary across projects based on factors such as statutory requirements, programs, and board policies. In general, approval milestones for the most common types of projects are as follows: -Loans and Grants above $250,000: the date of approval by ESD’s Board of Directors; -Loans and Grants $250,000 and under: the date of approval by ESD’s Commissioner; -Excelsior Tax Credits: the date of the project’s official admittance into the program; -START-UP: date of the business’ approval by the ESD Commissioner; -Life Sciences Tax Credit Program: January 1st of the tax credit benefit year; -Employee Training Incentive Program: the date of the project’s conditional admittance into the program; -Film Production Tax Credit: the date the initial application is approved -Post-Production Tax Credit: the date the initial application is approved; -Musical & Theatrical Tax Credit: the date the initial application is approved; -Commercial Tax Credit Program: January 1st of the tax credit benefit year; -Restaurant Return-To-Work Tax Credit Program: the start of the employment eligibility period. HCR: The date the award was made. NYPA: Month, day, year that the project becomes official. NYSERDA: Date in which the project commences or is contracted.
    "end_date", -- DOS: Date that all financial assistance has been disbursed for the project. ESD: Date that all financial assistance has been disbursed for the project. HCR: Not applicable or not currently collected. NYPA: Date that all financial assistance has been disbursed for the project. NYSERDA: Date at which projects are completed.
    "assistance_type", -- DOS: Type of assistance the project is receiving, defined as either loan, grant, or tax credit. ESD: Type of assistance the project is receiving, defined as either loan, grant, or tax credit. HCR: Not applicable or not currently collected. NYPA: Type of assistance the project is receiving, defined as either loan, grant, or tax credit. NYSERDA: NYSERDA’s funding type classification.
    "assistance_amount", -- All: The total dollar value for the specific agency and project award. This does not include additional dollars awarded to the project from other state sources.  - START-UP: The Start Date is not necessarily the beginning of the ten-year tax-free period or the date of disbursement. There is typically a lag in the disbursement time period.
    "total_lead_agency_benefits_awarded", -- DOS: Total dollar amount of DOS assistance the project is receiving through the Downtown Revitalization Initiative ESD: Total amount of ESD assistance the project is receiving in dollars. - START-UP: The Start Date is not necessarily the beginning of the ten-year tax-free period or the date of disbursement. There is typically a lag in the disbursement time period. HCR: Amount of funding provided through the CDBG grant by HCR. NYPA: Total amount in dollars of the grant funding awarded by NYPA for the project described NYSERDA: Cumulative NYSERDA funding contracted through the most recent reporting period. Green Bank loans are omitted.
    "disbursements_to_date", -- All: The total dollars disbursed to the project, subject to a reporting lag.
    "does_this_project_include_additional_nys_benefits", -- All: Indicates whether the project is receiving benefits outside of the reporting agency, program and award amount. This field can include other investments from the reporting agency (Yes/No).
    "if_yes_what_is_the_additional_awarding_agency_name_", -- If "Yes" to "Does the project include additional NYS benefits?", the name of the additional agency providing funding, if applicable.
    "other_state_agency_funding_program", -- All: The program through which the other agency is providing funding, if applicable.
    "other_state_agency_funding_awarded", -- All: Dollar amount of funding from other agencies, if applicable.
    "is_the_project_also_receiving_benefits_from_a_local_industrial_", -- All: Is the Project also receiving benefits from a local Industrial Development Agency (IDA) (Yes / No)
    "if_applicable_from_which_ida_is_the_project_also_receiving_bene", -- All: Name of the Industrial Development Agency, if applicable
    "if_applicable_what_is_the_net_value_of_the_tax_benefits_awarded", -- DOS: Calculated as net exemptions provided minus any PILOT payments made over the life of the IDA exemption, if applicable ESD: Calculated as the market value of all payments that would have been due minus any payments made. HCR: Not applicable or not currently collected. NYPA: n/a if not applicable NYSERDA: Not applicable or not currently collected.
    "total_nys_investment", -- DOS: Total dollar amount of DOS and other known NYS agency investments ESD: Total dollar amount of the awarding agency and other known NYS agency investments. - START-UP: The Start Date is not necessarily the beginning of the ten-year tax-free period or the date of disbursement. There is typically a lag in the disbursement time period. HCR: The sum of HCR and other agency partner investment amounts. NYPA: Total amount in dollars, to include other NYS awards if known NYSERDA: Cumulative NYSERDA funding contracted through the most recent reporting period. Green Bank loans are omitted.
    "total_public_private_investment", -- DOS: The entire cost of the project including any non-DOS funding. This field can include estimates for private investment for Downtown Revitalization Initiative projects. ESD: The total project cost of the project including any non-ESD funding. - START-UP: The Start Date is not necessarily the beginning of the ten-year tax-free period or the date of disbursement. There is typically a lag in the disbursement time period. HCR: The sum of all project investment amounts. NYPA: NYS investment + private funding to meet total project costs NYSERDA: NYSERDA contracted funding plus cumulative reported leveraged funding through the most recent reporting period.
    "project_status", -- DOS: The current status of the project, either Active, Closed or Terminated ESD: The current status of the project, either Active, Closed or Terminated. Please note that all tax incentive programs are marked as "Closed" when the tax credit is issued, and the project is deemed completed by ESD. For the Excelsior Jobs Tax Credit Program, this will occur after the ten-year benefit schedule has concluded, and the project is completed. HCR: The project status. NYPA: The current status of the project, either Active, Closed or Terminated NYSERDA: The current status of the project, either Active or Closed (completed), or terminated.
    "compliant_with_terms_and_conditions", -- DOS: Indicates whether the project is currently considered by DOS to be in compliance, per the specific project requirements. Compliance is presented as Yes/No. ESD: Indicates whether the project is currently considered by ESD to be in compliance, per the specific project requirements. Compliance is presented as Yes/No. HCR: Not applicable or not currently collected. NYPA: Yes or No indication of whether the project is currently considered by NYPA to be in compliance with agreement terms. NYSERDA: Whether NYSERDA considers a project in compliance with project requirements.
    "reason_for_non_compliant_status", -- DOS: If project is non-compliant, the reason for such non-compliance is provided. If a project is in compliance, this field will display “N/A”. ESD: If project is non-compliant, the reason for such non-compliance is provided. If a project is in compliance, this field will display “N/A”. HCR: Not applicable or not currently collected. NYPA: If project is non-compliant, the reason for such non-compliance is provided. If a project is in compliance, this field will display “N/A”. NYSERDA: If project is non-compliant: the reason for non-compliance. If a project is compliant: N/A
    "penalties_applied", -- DOS: Indicates whether the project has had penalties applied. Penalties Applied is presented as Yes/No. If a project is in compliance, this field will display “N/A”. ESD: Indicates whether the project has had penalties assessed. Penalties Assessed is presented as Yes/No. HCR: Not applicable or not currently collected. NYPA: Indicates whether the project has had penalties assessed. Penalties Assessed is presented as Yes/No. NYSERDA: Whether NYSERDA has assessed penalties for a project.
    "rationale_for_termination", -- All: If a project is terminated, the reason for such termination is provided.
    "is_this_a_job_creation_retention_project", -- All: Whether or not the project has a job retention and/or job creation commitment.
    "total_employees_at_the_site_ftes_", -- DOS: Total Employees (Full-time equivalent) at the assisted entity at the time of award. If grantee is a government entity, this field will display "N/A". ESD: Total Employees (full-time equivalent) at the assisted entity at the time of award. Full-time equivalent is defined as one full-time employee or two part-time, permanent, private-sector employees on the recipient’s payroll, who have worked at the project location for a combined minimum of thirty-five hours per week for not less than 4 consecutive weeks (specific time dictated by ESD program funding) and who are entitled to receive the usual and customary fringe benefits extended by the recipient to other employees with comparable rank and duties. Note: “full-time equivalent” may be defined slightly differently across projects and programs. HCR: Not applicable or not currently collected.    NYPA: Total number of employees at the site represented as Full Time Equivalents (FTEs) NYSERDA: Not applicable or not currently collected. 
    "total_employees_at_the_site_pts_", -- DOS: Total Employees (Part-time) at the assisted business, at the time of award. If grantee is a government entity, this field will display "N/A". ESD: Total employees (part-time) at the assisted entity at the time of award. Part-time is defined as an employee who works less than 35 hours per week for not less than four consecutive weeks.  HCR: Not applicable or not currently collected. NYPA: Not applicable or not currently collected. NYSERDA: Not applicable or not currently collected.
    "total_employees_at_the_site_project_hires_in_ftes_", -- DOS: Total FTE Project Hires. If grantee is a government entity, this field will display "N/A". ESD: Total Employees (full time project hires) at the assisted entity at the time of award. Total number of individual hires (head count), not full-time equivalents. “Project hires” are associated with specific programs, namely the Film Production Tax Credit, Post-Production Tax Credit, and the Musical & Theatrical Tax Credit which result in hires for the length of the project. HCR: Not applicable or not currently collected. NYPA: Not applicable or not currently collected. NYSERDA: Not applicable or not currently collected.
    "total_employees_at_the_site_project_hires_in_pt_workers_only_", -- DOS: Total PT Project Hires. If grantee is a government entity, this field will display "N/A". ESD: Total Employees (part time project hires) at the assisted entity at the time of award. Total number of individual hires (head count). “Project hires” are associated with specific programs, namely the Film Production Tax Credit, Post-Production Tax Credit, and the Musical & Theatrical Tax Credit which result in hires for the length of the project. HCR: Not applicable or not currently collected. NYPA: Not applicable or not currently collected. NYSERDA: Not applicable or not currently collected.
    "job_creation_commitments_ftes", -- DOS: Total number of new jobs (Full-time equivalent) that a project is committed to creating, when applicable. Projects that do not have direct job commitments will display "N/A" for this field. ESD: Total number of new jobs (FTEs) that a project is ultimately committed to creating, when applicable. HCR: Job creation commitments associated with the award (full-time equivalent employees). NYPA: Total number of new jobs that a project is required to create. NYSERDA: Not applicable or not currently collected.
    "job_creation_commitments_pts_", -- DOS: Total number of new jobs (Part-time) that a project is ultimately committed to creating, when applicable. Projects that do not have direct job commitments will display "N/A" for this field. ESD: Total number of new jobs (part-time) that a project is ultimately committed to creating, when applicable. Part time is defined as an employee who works less than 35 hours per week for not less than four consecutive weeks. HCR: Job creation commitments associated with the award (part time employees). NYPA: Total number of new part time jobs a project is required to create. NYSERDA: Not applicable or not currently collected.
    "job_retention_commitments_ftes_", -- DOS: Total number of new jobs (Full-time equivalent) that a project is committed to retaining, when applicable. Projects that do not have direct job commitments will display "N/A" for this field. ESD: Total number of existing jobs (FTEs) that a project is currently documented as having retained, when applicable. “Jobs Retained To-Date” can be defined slightly differently by different programs. HCR: Job retention commitments associated with the award (full-time equivalent employees). NYPA: Total number of jobs a project is committed to retaining. NYSERDA: Not applicable or not currently collected.
    "job_retention_commitments_pts_", -- DOS: Total number of new jobs (Part-time) that a project is committed to retaining, when applicable. Projects that do not have direct job commitments will display "N/A" for this field. ESD: Total number of jobs (part-time) that a project is committed to retaining, when applicable. Part time is defined as an employee who works less than 35 hours per week for not less than four consecutive weeks. HCR: Job retention commitments associated with the award (part time employees). NYPA: Total number of part time jobs a project is required to retain. NYSERDA: Not applicable or not currently collected.
    "jobs_created_to_date_ftes", -- DOS: Total number of jobs (Full-time equivalent) that a project is currently documented as having created, when applicable. Note that many projects, such as community investments into public spaces, may not report jobs created even though they may contribute to economic development. Projects that do not have direct job commitments will display "N/A" for this field. ESD: Total number of jobs (FTEs) that a project is currently documented as having created, when applicable. Note that many projects, such as community investments into public spaces, may not report jobs created even though they may contribute to economic development, and that many projects have several years to reach their total Job Creation Commitments before they are considered not to be in compliance. Jobs may in some cases count multiple jobs added together to create one “full-time equivalent” job and may be defined slightly differently across projects and programs. HCR: Jobs created to date (full-time equivalent employees). NYPA: Total number of jobs (FTEs) at time of most recent compliance report. NYSERDA: Not applicable or not currently collected.
    "jobs_created_to_date_pts_", -- DOS: Total number of jobs (Part-time) that a project is currently documented as having created, when applicable. Note that many projects, such as community investments into public spaces, may not report jobs created even though they may contribute to economic development. Projects that do not have direct job commitments will display "N/A" for this field. ESD: Total number of jobs (part-time) that a project is currently documented as having created, when applicable. Part time is defined as an employee who works less than 35 hours per week for not less than four consecutive weeks. Note that many projects, such as community investments into public spaces, may not report jobs created even though they may contribute to economic development, and that many projects have several years to reach their total Job Creation Commitments before they are considered not to be in compliance. Jobs may in some cases count multiple jobs added together to create one “full-time equivalent” job and may be defined slightly differently across projects and programs. HCR: Jobs created to date (part time employees). NYPA: Total number of part time jobs at time of most recent compliance report. NYSERDA: Not applicable or not currently collected.
    "jobs_retained_to_date_ftes", -- DOS: Total number of existing jobs (Full-time equivalent) that a project is currently documented as having retained, when applicable. “Jobs Retained To-Date” can be defined slightly differently by different programs. Projects that do not have direct job commitments will display "N/A" for this field. ESD: Total number of existing jobs that a project is currently documented as having retained, when applicable. “Jobs Retained To-Date” can be defined slightly differently by different programs. HCR: Jobs retained to date (full-time equivalent employees). NYPA: Total number of jobs (FTEs) retained at time of most recent compliance report. NYSERDA: Not applicable or not currently collected.
    "jobs_retained_to_date_pts_", -- DOS: Total number of existing jobs (Part-time) that a project is currently documented as having retained, when applicable. “Jobs Retained To-Date” can be defined slightly differently by different programs. Projects that do not have direct job commitments will display "N/A" for this field. ESD: Total number of existing jobs (Part-time) that a project is currently documented as having retained, when applicable. “Jobs Retained To-Date” can be defined slightly differently by different programs. Part time is defined as an employee who works less than 35 hours per week for not less than four consecutive weeks. HCR: Total number of existing jobs (Part-time) that a project is currently documented as having retained, when applicable. “Jobs Retained To-Date” can be defined slightly differently by different programs. Projects that do not have direct job commitments will display "N/A" for this field. NYPA: Total number of part time jobs retained at time of most recent compliance report. NYSERDA: Not applicable or not currently collected.
    "project_hires_ftes", -- DOS: Total number of individual project hires (Full-time equivalent). Projects that do not have direct job commitments will display "N/A" for this field. ESD: Total number of individual hires (head count), not full-time equivalents.  “Project hires” are associated with specific programs, namely the Film Production Tax Credit, Post-Production Tax Credit, Commercial Tax Credit Program and the Musical & Theatrical Tax Credit which result in hires for the length of the project.  HCR: Total number of individual project hires (Full-time equivalent). Projects that do not have direct job commitments will display "N/A" for this field. NYPA: Total number of temp or seasonal employees hired as of the most recent compliance report. NYSERDA: Not applicable or not currently collected.
    "project_hires_pts_", -- DOS: Total number of individual project hires (Part-time). Projects that do not have direct job commitments will display "N/A" for this field. ESD: Total number of individual hires (head count), working less than 35 hours per week. “Project hires” are associated with specific programs, namely the Film Production Tax Credit, Post-Production Tax Credit, and the Musical & Theatrical Tax Credit which result in hires for the length of the project. HCR: Total number of individual project hires (Part-time). Projects that do not have direct job commitments will display "N/A" for this field. NYPA: Total number of part time temp or seasonal employees hired as of most recent compliance report. NYSERDA: Not applicable or not currently collected.
    "nys_mwbe_utilization_goal_applied", -- DOS: The percent utilization goal applied ESD: The percent utilization goal applied to either the grant amount or the portion of the grant that has potential opportunities for certified mwb spending. HCR: The percent utilization goal applied NYPA: NYPA’s assigned goal is 30%: separated into 18% Minority, 12% Woman Business Enterprises NYSERDA: Not applicable or not currently collected.
    "current_mwbe_utilization_rate", -- DOS: The percent of the goal amount that has been utilized ESD: The percent of the goal amount that has been satisfied to date. HCR: The percent of the goal amount that has been utilized NYPA: Percentage achieved at time of final disbursement NYSERDA: Not applicable or not currently collected.
    "is_this_a_legacy_project", -- All: Legacy projects are defined as projects initiated prior to October 1, 2022, and therefore were not subject to the additional data fields which were added pursuant to the DOEI enhancements as part of the SFY 2022-2023 NYS Budget.  Such projects may have additional blank fields, as the mandate for recipients to report on those fields was not yet in place
    "contact", -- All: Agency email address for database related inquiries.
    "original_recipient", -- All: The name of Original organization receiving economic assistance from reporting agency at the time of ESD Award.
    "original_jobs_created_fte_", -- All: Original Total Employees (full-time equivalent) at the assisted entity at the time of ESD Award.
    "original_jobs_retained_fte", -- All: Original Total Employees (part-time equivalent) at the assisted entity at the time of ESD Award.
    "original_total_public_private_investment", -- All: The Original total project cost of the project, including any non-ESD funding, at the time of ESD Award.
    "original_assistance_amount" -- All: The Original total dollar value for the specific agency and project award at the time of ESD Award. This does not include additional dollars awarded to the project from other state sources.
FROM
    "ny-gov/database-of-economic-incentives-26ei-n4eb:latest"."database_of_economic_incentives"
LIMIT 100;

Connecting to the DDN is easy. All you need is an existing SQL client that can connect to Postgres. As long as you have a SQL client ready, you'll be able to query ny-gov/database-of-economic-incentives-26ei-n4eb with SQL in under 60 seconds.

Query Your Local Engine

Install Splitgraph Locally
bash -c "$(curl -sL https://github.com/splitgraph/splitgraph/releases/latest/download/install.sh)"
 

Read the installation docs.

Splitgraph Cloud is built around Splitgraph Core (GitHub), which includes a local Splitgraph Engine packaged as a Docker image. Splitgraph Cloud is basically a scaled-up version of that local Engine. When you query the Data Delivery Network or the REST API, we mount the relevant datasets in an Engine on our servers and execute your query on it.

It's possible to run this engine locally. You'll need a Mac, Windows or Linux system to install sgr, and a Docker installation to run the engine. You don't need to know how to actually use Docker; sgrcan manage the image, container and volume for you.

There are a few ways to ingest data into the local engine.

For external repositories, the Splitgraph Engine can "mount" upstream data sources by using sgr mount. This feature is built around Postgres Foreign Data Wrappers (FDW). You can write custom "mount handlers" for any upstream data source. For an example, we blogged about making a custom mount handler for HackerNews stories.

For hosted datasets (like this repository), where the author has pushed Splitgraph Images to the repository, you can "clone" and/or "checkout" the data using sgr cloneand sgr checkout.

Cloning Data

Because ny-gov/database-of-economic-incentives-26ei-n4eb:latest is a Splitgraph Image, you can clone the data from Spltgraph Cloud to your local engine, where you can query it like any other Postgres database, using any of your existing tools.

First, install Splitgraph if you haven't already.

Clone the metadata with sgr clone

This will be quick, and does not download the actual data.

sgr clone ny-gov/database-of-economic-incentives-26ei-n4eb

Checkout the data

Once you've cloned the data, you need to "checkout" the tag that you want. For example, to checkout the latest tag:

sgr checkout ny-gov/database-of-economic-incentives-26ei-n4eb:latest

This will download all the objects for the latest tag of ny-gov/database-of-economic-incentives-26ei-n4eb and load them into the Splitgraph Engine. Depending on your connection speed and the size of the data, you will need to wait for the checkout to complete. Once it's complete, you will be able to query the data like you would any other Postgres database.

Alternatively, use "layered checkout" to avoid downloading all the data

The data in ny-gov/database-of-economic-incentives-26ei-n4eb:latest is 0 bytes. If this is too big to download all at once, or perhaps you only need to query a subset of it, you can use a layered checkout.:

sgr checkout --layered ny-gov/database-of-economic-incentives-26ei-n4eb:latest

This will not download all the data, but it will create a schema comprised of foreign tables, that you can query as you would any other data. Splitgraph will lazily download the required objects as you query the data. In some cases, this might be faster or more efficient than a regular checkout.

Read the layered querying documentation to learn about when and why you might want to use layered queries.

Query the data with your existing tools

Once you've loaded the data into your local Splitgraph Engine, you can query it with any of your existing tools. As far as they're concerned, ny-gov/database-of-economic-incentives-26ei-n4eb is just another Postgres schema.

Related Documentation:

Loading...