Today, after a year of internal development and incubation, we are pleased to open source Aleph, an analytics platform for Redshift.

At Lumosity, we use data in a number of ways – you can learn more about data at Lumosity in this Quora post from one of our Data Scientists – and we have dedicated Data Science and Analytics teams. Still, because data plays an important role in making key product, research and business decisions across the organization, we wanted to create a tool that would empower people on other teams to analyze data relevant to their work more efficiently and easily.

While there are many amazing products designed for SQL exploration, we focused our efforts on creating a new tool that would, we hoped, “teach a man to fish.” Essentially, we needed something that would multiply not only our data insights, but also empower more people to generate those insights – a tool that would both facilitate querying and act as a guide to those who wanted to learn to query for themselves.

To that end, we developed Aleph, which focuses on aggregating institutional know-how associated with data querying. The name comes from the eponymous Borges short story, in which a single point connects to all other points: at Lumosity, Aleph has become the entry point to our manifold data. Aleph is also a central repository for institutional knowledge, allowing users to collaborate and iteratively improve on queries. Query versions can be maintained in GitHub so others may better understand the path and genealogy of each investigation. Importantly, Aleph has robust data access controls with table-level granularity, allowing users to research the topics relevant and necessary for their domain while also ensuring that sensitive information is protected and access is controlled appropriately.

Aleph is now in wide use across Lumosity. From our finance team to our PMs, we’ve seen tremendous value both created by the tool and in the tool itself. In open sourcing Aleph, we’re excited to see how our friends at other companies find benefits in adopting a “teach a man to fish” approach to data, too.

Below you’ll find a comprehensive list of Aleph’s Key Features. If you have any questions, concerns, or general feedback, be sure to share your thoughts with us on GitHub or Twitter.

aleph_bannner

Get Started

Watch a quick overview of basic functionality.

Or, if you want to connect to your own Redshift cluster and have Docker - do this:

docker run -ti -p 3000:3000 lumos/aleph-playground /bin/bash -c "aleph setup_minimal -H {host} -D {db} -p {port} -U {user} -P  {password}; redis-server & aleph run_demo"


Find out more & contact us.

Feature List

  • Query Redshift
  • Understand the Structure of Your Data
    • Search metadata: schemas, tables, and columns
    • Ability to document columns with descriptions
  • Query Development: Understand, Learn, and Share
    • Browse sortable index of queries
    • Query search by full text or attributes (author, title, or tags)
    • Query versions backed by github - access to commit diffs, and history
    • Annotate query versions with comments
    • Share queries and groups of queries by link (search terms in the URL)
  • SQL Writing Tools
    • Specialized query writing/execution REPL view
    • “Smart” Autocomplete based on metadata and SQL structure
    • Snippets with autocomplete integration
    • Query parameters
  • Rich, Customizable C3 Visualizations
  • Fast, Efficient Query Execution
    • Streaming execution - low memory footprint and almost immediate results for certain queries
    • Arbitrarily large Results backed by Aws S3
    • Download as CSV
  • Business Intelligence Alerting
    • Write queries as alerts
    • Email interested parties when metrics reach a configurable threshold
  • Access Control (optional)
    • SAML (e.g. Okta)
    • Rails Admin
    • Configurable to be granular on the table level