➡️Importing data via API

Step-by-Step guide for Inserting data in Engines

Prerequisites

  • Register with Sepana

  • Have at least read-write access to the Engine. Sepana automatically creates an API key with owner access. The Engine creator can use that key for inserting the data. Alternatively, they can create another key with "Read-Write" or "Admin" ACL and use it. If you are not the creator of the Engine, you will need someone (owner or admin of the Engine) to assign you the API key access. (Discussed here)

  • Data in JSON form. Currently, our API supports indexing data in JSON form. You can manually upload other supported data types(CSV, JSON, TSV) via the developer dashboard.

Steps

Fetch API key

Navigate to the "Engine View" and Click on API Keys.

You should see the API keys accessible to you along with their permissions. As you can see, one of the keys has Admin rights while the other has Read_only rights. So, for this particular Engine, we can use the "Share API" key because it has "Admin" rights.

ENGINE_ID = "24c52517-b45f-497b-8e7a-a724d3d7594b"
API_KEY = "IlOgXKj-pCzcdhIF9po3y39q7O3l5UTr6Y5QYdMzuR7hwAtqGryW_Q"

Convert Documents to JSON format

Sepana REST API for insertion currently only supports JSON documents. Therefore, this step requires converting our input documents to JSON format. A maximum of 500 documents (or objects) can be inserted in a single API call.

Insert

Now, that we have the Engine ID, API key, and documents in JSON format. We can call the insert data endpoint. For this example, we will be inserting two documents.

Request
curl --location --request POST 'https://api.sepana.io/v1/engine/insert_data' \
--header 'x-api-key: IlOgXKj-pCzcdhIF9po3y39q7O3l5UTr6Y5QYdMzuR7hwAtqGryW_Q' \
--header 'Content-Type: application/json' \
--data-raw '{
    "engine_id": "24c52517-b45f-497b-8e7a-a724d3d7594b",
    "docs": [
        {
      "title": "Tesla Shanghai Capacity Will Exceed 1 Million Cars In 2022, BYD Blade Batteries Coming",
      "link": "https://cleantechnica.com/2022/02/14/tesla-shanghai-capacity-will-exceed-1-million-cars-in-2022-byd-blade-batteries-coming/",
      "keywords": [
        "Electric Cars",
        "Uncategorized",
        "BYD",
        "BYD Blade Battery",
        "Tesla",
        "Tesla batteries",
        "Tesla factories",
        "Tesla Giga Shanghai",
        "Tesla Gigafactories"
      ],
      "creator": [
        "Steve Hanley"
      ],
      "video_url": null,
      "description": "Tesla'\''s Giga Shanghai is expanding and will soon be able to manufacture more than 1 million cars a year.",
      "content": null,
      "pubDate": "2022-02-14 05:28:46",
      "full_description": "According to reports from CnEVPost , the manufacturing capacity of the Tesla Gigafactory in Shanghai is expected to increase to more than 1 million cars a year in the near future. In addition, it reports that BYD will supply about 10 GWh of its LFP Blade batteries a year to Tesla — enough to power more than 200,000 cars. Tesla Giga Shanghai Growth Tesla began a major expansion of its Gigafactory in Shanghai late last year. Currently, it is capable of building 450,000 cars a year . That number is expected to rise to more than 1 million cars a year once the expansion in complete. The enlarged facility is scheduled to begin production in April of this year, according to sources. Tesla said when the expansion project began it would achieve capacity increases by increasing work hours, increasing the number of employees, and increasing the frequency of material turnover. It expects to add 4,000 employees to its payroll, bringing the total workforce at the Shanghai factory to 19,000. This month, on the first day of the Chinese New Year, Tesla posted openings for 1,523 new positions in 11 categories including sales and delivery, service, R&D, manufacturing, infrastructure, and supply chain. The production line optimization will focus on expanding the press shop, body shop, paint shop, final assembly shop, and logistics operation center. BYD Blade Battery Order Rumors have been circulating for some time that Tesla China has placed a large order for BYD Blade batteries . Recently, a source within China posted on social media that BYD will begin mass production of its C112F LFP batteries in March and that Teslas fitted with the new batteries are scheduled to begin shipping from Giga Shanghai in the second quarter. Later, the post was deleted, apparently not because the information was inaccurate but because BYD wished to make the official announcement itself. If Tesla does indeed expand its Giga Shanghai capacity to 1 million cars, BYD could be the battery supplier for about 20% of them. Cars built in China won’t be reserved exclusively for in-country sales but will be shipped to other markets in Asia. The standard range versions of the Model 3 and Model Y currently produced at Tesla’s Shanghai use LFP batteries manufactured by CATL at its factory located just a few blocks away from Giga Shanghai . Tesla has indicated that LFP batteries will be used in both the standard and long range versions of its Model 3 and Model Y cars manufactured in the US and in Europe. BYD already has a US battery factory that manufactures LFP batteries. We leave it to our readers to connect the dots. Appreciate CleanTechnica’s originality? Consider becoming a CleanTechnica Member, Supporter, Technician, or Ambassador — or a patron on Patreon . Advertisement Have a tip for CleanTechnica, want to advertise, or want to suggest a guest for our CleanTech Talk podcast? Contact us here . Twitter LinkedIn Facebook",
      "image_url": null,
      "source_id": "cleantechnica",
      "country": [
        "united states of america"
      ],
      "category": [
        "business"
      ],
      "language": "english"
    },
    {
      "title": "Elon Musk asegura que la '\''verdadera'\'' deuda de EU es de al menos 60 billones de dólares",
      "link": "https://www.milenio.com/negocios/elon-musk-deuda-eu-60-billones-dolares",
      "keywords": null,
      "creator": [
        "Milenio Digital"
      ],
      "video_url": null,
      "description": "El director ejecutivo de Tesla y SpaceX, Elon Musk, estimó que la deuda de Estados Unidos en tres veces mayor a su economía. ?La verdadera deuda nacional, incluidos los derechos no financiados, es de al menos 60 billones de dólares, aproximadamente tres veces el tamaño de toda la economía de los Estados Unidos. Algo tiene que ceder.?, señaló el multimillonario a través de su cuenta de Twitter.True national debt, including unfunded entitlements, is at least $60 trillion ? roughly three times the size of the entire US economy. Something has got to give.? Elon Musk (@elonmusk) February 11, 2022 Esto como parte de los comentarios en respuesta a una publicación del medio satírico The Babylon Bee, en donde señaló que supuestamente el presidente estadunidense, Joe Biden, planeaba pagar la deuda pública apostando ?doble o nada? en el equipo de futbol de los Cincinnati Bengals en el Super Bowl. Según el Departamento del Tesoro de Estados Unidos el total de la deuda de dicho país ya supera los 30 billones de dólares, dio a conocer el martes pasado.El endeudamiento del gobierno se aceleró durante la pandemia de covid-19, como parte del gasto del gobierno estadunidense para amortiguar el golpe económico de la crisis. La deuda pública se ha disparado en alrededor 7 billones desde finales de 2019.m{1861862}RM?",
      "content": null,
      "pubDate": "2022-02-14 05:05:43",
      "full_description": null,
      "image_url": "http://www.milenio.com/uploads/media/2022/02/13/elon-musk-estimo-deuda-unidos.jpeg",
      "source_id": "milenio",
      "country": [
        "mexico"
      ],
      "category": [
        "top"
      ],
      "language": "spanish"
    }
    ]
}'
Response
{
    "job_id": "sepana_index_7926e37c-1e2e-4644-9e39-b0a6e83e9f87",
    "engine_id": "24c52517-b45f-497b-8e7a-a724d3d7594b",
    "request_timestamp": "12/02/2022, 13:01:01",
    "status": "queued"
}

Upon calling the insertion endpoint, the request is queued and handled by the system asynchronously. You can check the status of the job using Job Status endpoint.

Last updated