Ruby SDK

The intosql gem wraps the IntoSQL API for Ruby, one of the official SDKs. One call uploads a file, starts the conversion and waits for the result. The gem has no dependencies and uses the standard library's net/http, json and openssl.

Pro: The API is part of IntoSQL Pro. Make an API key in Settings, API keys and keep it out of your source code.

Install#

gem install intosql

Or add it to your Gemfile with bundle add intosql. The gem runs on Ruby 3.1 or newer.

Create a client#

require "intosql"

intosql = IntoSQL::Client.new(api_key: ENV.fetch("INTOSQL_API_KEY"))

The constructor raises ArgumentError if api_key is empty or nil.

The client never sends your key to another origin. If a pagination link points somewhere other than base_url's origin, the call raises IntoSQL::Error instead. Every request carries User-Agent: intosql-ruby/<version>.

Transport#

A transport is any object with call(method:, url:, headers:, body:) that returns an IntoSQL::Response (status, headers with lowercase names, body as a string) and raises IntoSQL::ConnectionError when the connection fails. body is nil, a JSON string, or the IO of an upload.

class Recorder
  def call(method:, url:, headers:, body: nil)
    IntoSQL::Response.new(status: 200, headers: {}, body: '{"data":{"limit":1,"used":0,"left":1,"period_start":"2026-10-01T00:00:00+00:00"}}')
  end
end

IntoSQL::Client.new(api_key: "test", transport: Recorder.new).usage.left # => 1

Convert a file#

convert(file, spec:, name: nil, bytes: nil) does the whole flow and returns the finished conversion.

require "intosql"

intosql = IntoSQL::Client.new(api_key: ENV.fetch("INTOSQL_API_KEY"))

conversion = intosql.convert("ventas.xlsx", spec: IntoSQL.spec(dialect: "postgresql", table: "ventas"))

puts conversion.report[:rows][:converted], conversion.output.url

What happens, in order:

1. Open. POST /conversions with name, the file's size in bytes and spec. The answer has an upload address.
2. Upload. The file is sent with a PUT to that address, with the headers the API gave, as it is. Nothing is read into memory first.
3. Start. POST /conversions/{id}/start.
4. Wait. The conversion is read until its status is done or failed.

file is a path (String or Pathname), opened and closed for you, or an IO such as a File or StringIO, streamed from where it stands. The size comes from the file or the IO's size; an IO without one, such as a pipe or a socket, needs bytes: with its exact size, since the conversion only starts once the whole file has arrived. To upload a string of bytes, wrap it in a StringIO.

require "net/http"

uri = URI("https://example.com/exports/pedidos.json")

Net::HTTP.get_response(uri) do |response|
  reader, writer = IO.pipe
  Thread.new do
    response.read_body { |chunk| writer.write(chunk) }
    writer.close
  end

  intosql.convert(
    reader,
    name: "pedidos.json",
    bytes: Integer(response["Content-Length"]),
    spec: IntoSQL.spec(dialect: "sqlite")
  )
end

name helps tell the file's format, and the downloaded result is named after it: ventas.xlsx downloads as ventas.sql. It defaults to the file's base name, and is required for an IO without a path. Set table in the spec to choose the table's name.

A MySQL or MariaDB dump (.sql, .sql.gz) converts to PostgreSQL like any file. job_report reads the report with each of its tables:

conversion = intosql.convert("shop.sql.gz", spec: IntoSQL.spec(dialect: "postgresql"))

conversion.job_report.tables.each do |table|
  puts "#{table.name}: #{table.rows} rows"
end

For a dump, source says where it comes from, tables holds each table's rows, columns, fixes and problems, and not_converted lists the views, triggers, routines and events it did not carry. For other files, source is nil and both lists are empty.

How it waits#

convert, convert_url and conversions.wait read the conversion again and again with growing pauses: half a second, then 1, 2 and 4 seconds, then every 5 seconds. They return when the status is done, raise IntoSQL::ConversionFailedError when it is failed, and IntoSQL::ConversionCanceledError when it was canceled.

Giving up only stops the waiting. The conversion keeps running on IntoSQL's servers, and you can read it later by its id.

conversion = intosql.convert(
  "clientes_2024.xlsx",
  spec: IntoSQL.spec(dialect: "mysql"),
  timeout: 10 * 60
)

Convert a file by URL#

convert_url(url, spec:, name: nil) asks IntoSQL to download the file from a public HTTPS address, then waits the same way. There is no upload step.

conversion = intosql.convert_url(
  "https://example.com/exports/ventas.csv",
  spec: IntoSQL.spec(dialect: "mysql")
)

It takes spec:, an optional name: (otherwise the name comes from the server's Content-Disposition header or the address), and the wait options. The address must be HTTPS and reachable from the internet; anything else is refused with invalid_url. If the download fails, the conversion ends as failed.

Download the result#

When status is done, output.url downloads the result. The address works for ten minutes. After that, read the conversion again to get a fresh one.

require "net/http"

fresh = intosql.conversions.retrieve(conversion.id)
uri = URI(fresh.output.url)

Net::HTTP.start(uri.host, uri.port, use_ssl: true) do |http|
  http.request_get(uri) do |response|
    File.open("ventas.sql", "wb") { |file| response.read_body { |chunk| file.write(chunk) } }
  end
end

Send no Authorization header to output.url; the address carries its own signature. Files are deleted 24 hours after the conversion finishes, or right away with conversions.delete.

Build a spec#

IntoSQL.spec(...) returns a full conversion spec, a hash with symbol keys, with IntoSQL's defaults. Only dialect: is required.

request = IntoSQL.spec(
  dialect: "sql_server",
  table: "clientes",
  schema: "ventas",
  sheet: "Clientes 2024",
  batch_rows: 1000
)

The rest is fixed by the helper: format, encoding, delimiter and header row detected from the file; column names in snake_case; INSERT statements (style: "inserts"); types inferred from the first 10,000 rows; date order detected; leading zeros kept; no per-column overrides. For anything else (bulk output, a date order, a locale, columns renamed, kept as text or left out), change the hash it returns or write the spec yourself. Every field is in Conversion spec.

request = IntoSQL.spec(dialect: "postgresql", table: "pedidos")

request[:output][:style] = "bulk"
request[:inference][:date_order] = "day_first"
request[:columns] = [{ index: 0, as_text: true }]

Conversions#

intosql.conversions exposes each step on its own, for when you want to upload from somewhere else, start later, or not wait.

Step by step#

opened = intosql.conversions.create(
  name: "ventas.xlsx",
  bytes: File.size("ventas.xlsx"),
  spec: IntoSQL.spec(dialect: "mariadb")
)

intosql.conversions.upload(opened, "ventas.xlsx")
intosql.conversions.start(opened.id)

# Later, or from a webhook handler:
conversion = intosql.conversions.retrieve(opened.id)

The upload address from create works for an hour. upload takes the same files as convert and raises IntoSQL::Error if the conversion has no upload address (one read with retrieve never does), and IntoSQL::APIError with code upload_failed if storage doesn't answer with a 2xx.

List#

list returns a lazy Enumerator over every conversion on the account, newest first. It fetches a page at a time (50, or limit: from 1 to 100) and follows links.next until there are no more pages. status: keeps only conversions in that status.

intosql.conversions.list(status: "failed").each do |conversion|
  puts conversion.id, conversion.error&.code, conversion.name
end

Stop early with break, first(n) or lazy; no further pages are fetched.

Cancel#

cancel(id) stops a conversion that hasn't ended and deletes its files. It stays listed as canceled and doesn't count toward the month. Canceling it again answers the same; canceling one that already ended raises IntoSQL::APIError with code already_finished.

intosql.conversions.cancel(conversion.id)

Send us a conversion#

When a file didn't convert well, or at all, create_case sends the conversion to IntoSQL so support for it gets added. Pass consent: true only when the person agreed to send it; anything else raises ArgumentError. share: "file" sends the file as well as the report; a failed conversion keeps its upload until it expires, so the file is still there to send.

intosql.conversions.create_case(
  conversion.id,
  consent: true,
  share: "file",
  note: "Exported from our ERP; the dates come out wrong."
)

Cases are private, kept 90 days, and then deleted. The privacy policy says how they are handled.

Delete#

delete(id) removes the conversion's input and output files now instead of after 24 hours. Reading it afterwards answers not_found.

intosql.conversions.delete(conversion.id)

Usage#

usage returns the bytes the plan includes this month, used and left.

usage = intosql.usage

puts usage.limit, usage.used, usage.left, usage.period_start

Failed conversions don't count toward used.

Verify webhooks#

With a webhook address saved in Settings, API keys, IntoSQL sends a signed POST when a conversion is done, failed or canceled. IntoSQL.parse_webhook checks the IntoSQL-Signature header against the raw body and returns the typed event; it raises IntoSQL::SignatureError when the signature doesn't match.

class IntoSQLWebhooksController < ApplicationController
  skip_forgery_protection

  def create
    event = IntoSQL.parse_webhook(
      request.raw_post,
      request.headers["IntoSQL-Signature"].to_s,
      ENV.fetch("INTOSQL_WEBHOOK_SECRET")
    )

    if event.type == "conversion.done"
      # event.data is the conversion; event.id is the same on every retry.
    end

    head :no_content
  rescue IntoSQL::SignatureError
    head :bad_request
  end
end

IntoSQL.verify_webhook(body, header, secret) does the check alone and returns true or false. Both refuse a call when the header is malformed, the signature doesn't match, or the timestamp is more than 300 seconds from now. They take tolerance: to change that window and now: (Unix seconds) to fix the current time in tests. Pass the body exactly as received, before any JSON parsing. The signatures are compared in constant time. The payload and retries are in Webhooks.

Errors#

Everything the gem raises on purpose inherits from IntoSQL::Error.

• IntoSQL::APIError: the API answered with an error. It has status (the HTTP status), code (stable between versions), message (for people), fields (per field messages when the code is invalid_request, otherwise {}) and request_id, to quote when you write to us. If the answer has no error body, code is http_error.
• IntoSQL::ConversionFailedError: the conversion ended as failed. It has id, code (stable: ours, such as timed_out, or the engine's, such as encrypted_file), problems (what the engine found) and the explanation as its message.
• IntoSQL::ConversionCanceledError: the conversion was canceled while it was awaited. It has id.
• IntoSQL::TimeoutError: waiting passed timeout:. It has id and the last status.
• IntoSQL::ConnectionError: the connection kept failing after the retries. The network error is its cause.
• IntoSQL::SignatureError: a webhook call's signature doesn't match.

begin
  intosql.convert("ventas.xlsx", spec: IntoSQL.spec(dialect: "oracle"))
rescue IntoSQL::ConversionFailedError => e
  warn "Conversion #{e.id} failed (#{e.code}): #{e.message}"
rescue IntoSQL::APIError => e
  if e.code == "over_quota"
    warn "This month's conversions are used up."
  else
    warn [e.status, e.code, e.message, e.fields].inspect
  end
end

Every code, its status and how to fix it is in Errors.

Retries and idempotency#

The client retries a request when the connection fails, when the API answers rate_limited, too_many_active or request_in_progress, on any 5xx, and on 408 and 425 from the network in between, up to max_retries times (2 by default). It waits for Retry-After when the answer has it, and otherwise half a second, then 1 second, doubling up to 8 seconds.

Every POST gets an Idempotency-Key header with a random UUID, and its retries reuse the same key. If a first attempt reached the API but its answer was lost, the retry gets that first answer back instead of opening a second conversion.

Not retried: the file upload to storage (a failed PUT raises upload_failed), already_started (the conversion is on its way; read it instead), and any other 4xx such as invalid_spec or over_quota.

Types#

The answers are structs with a reader for each field. spec and report stay hashes with symbol keys, as the API sends them; job_report reads report as a JobReport. Fields the API adds later are ignored, and statuses or codes the gem doesn't know arrive as plain strings.

Timestamps are ISO 8601 strings; parse them with Time.iso8601 when you need a Time. report is described in Conversions, and fixes in Fixes.