WARNING: As of June 2011, PyTables Pro does not exist anymore as a separate product, and its functionality has been merged in regular PyTables (BSD edition). This section remains here mainly because it is a nice reference of the capabilities being added to the new PyTables. More info in NewPyTables.


pytables-pro_logo.png

PyTables Professional Edition

PyTables Professional Edition (aka PyTables Pro) is a commercial and much enhanced version of the Free Source software PyTables Standard Edition. In brief, PyTables Pro is ready for getting the most of the hardware behind it, allowing to perform complex data analysis on datasets that are typically larger (and sometimes much larger) than your available memory. With typical usage, PyTables Pro can cope, while using similar resources, with datasets that can be up to 5x larger than what traditional databases would allow you.

PyTables Pro’s simple and elegant database design and operation capabilities reduce the need for database administrators. In addition, its outstanding test suite and years of accumulated experience, guarantee exceptionally high levels of quality and stability.

PyTables Pro is being actively used in a wide spectrum of fields, like aeronautics, drug discovery, financial analysis, telecommunications switching systems, data mining or statistical analysis, to name just a few, allowing to deal with extremely large datasets with maximum efficiency.


Main features

Performance of queries for different optimization levels

It comes with OPSI included

OPSI is a powerful and innovative indexing engine allowing PyTables Pro to perform really fast queries on arbitrarily large tables. Moreover, it offers a wide range of optimization levels for its indexes so that the user can choose the best one that suits her needs (more or less size, more or less performance). Indexation code also takes advantage of the vectorization capabilities of the NumPy and Numexpr packages to ensure really short indexing and search times.

By using OPSI it is possible to complete a query on a table in the order of ten thousand million (10,000,000,000) rows in a matter of hundredths of a second. Moreover, it can index columns in times between 1.5x and 25x shorter (depending on the table size and the desired optimization level) than other databases, while reducing the size of the indexes between 3x and 15x.

Performing queries in very large (indexed) tables, fast

The main reason behind this incredible performance and compactness is that OPSI is geared towards tables that are mostly used for read-only or append-only purposes, and this is the scenario where it absolutely shines. If the user needs to update or delete frequently the values of indexed columns, then OPSI will take much more time than other solutions to keep its indexes updated. So, for situations that don't require fast updates or deletions, OPSI is probably one of the best indexing engines available.

Last but not least, and thanks to OPSI, PyTables Pro provides the ability to sort arbitrarily large tables by a specific field.

For more information about this innovative indexing engine, see the OPSI capabilities section below. And, although OPSI has got many improvements since this was written, you may also want to have a look at the OPSI White Paper, as many details exposed there are still valid for the current version.

Improved cache implementation

PyTables Pro wears a fine-tuned LRU cache for both metadata (nodes) and regular data that lets you achieve maximum speed for intensive object tree browsing and during data reads and queries. It complements the already efficient cache present in HDF5, although this is more geared towards high-level structures that are specific to PyTables Pro and that are critical for achieving very high performance.

The PyTables Pro installer

Professional installer for Windows

No more need to chase across the web for missing pieces, or play the version numbers game. Both 32-bit and 64-bit flavors have been put at your disposal, so just download one single package, and you're ready to begin work. And, although all-in-one installers for other platforms are not available, PyTables Pro can be quickly deployed on these by using Python's distutils anyway.

Meant for production

Near 50,000 carefully designed test units (in 2.2 version) check every detail and feature of PyTables Pro. Besides, for every new version of the product, all the tests are verified to successfully pass for the most common platforms (Windows, Mac OS X, Linux, FreeBSD, all in both 32-bit and 64-bit flavors). This way, you can relax and concentrate your efforts in resolving your own problems (and not the underlying libraries).

Support from PyTables developers, support for PyTables developers

Every license purchased contributes to our development efforts, ensuring the future of the PyTables project. In addition, PyTables Pro site and developer licenses come with first-class, high priority support, that allows you to get help directly from PyTables developers and maximize your productivity.


Special capabilities implemented in OPSI

OPSI engine in PyTables Pro comes with a series of new and exciting features that cannot be found in other indexing engines. Among them you will find:

Size of indexes in PyTables Pro 2.1

Index creation time in PyTables Pro 2.1

Performance of queries for different compressors

Performance of queries (unsorted vs sorted tables)


With all these advanced features, consider what OPSI can do for accelerating the mining of your huge table, without wasting precious computer and man-time resources.

PyTablesPro (last edited 2011-09-03 06:57:39 by FrancescAlted)