WARNING: As of June 2011, PyTables Pro does not exist anymore as a separate product, and its functionality has been merged in regular PyTables (BSD edition). This section remains here mainly because it is a nice reference of the capabilities being added to the new PyTables. More info in NewPyTables.
PyTables Professional Edition
PyTables Professional Edition (aka PyTables Pro) is a commercial and much enhanced version of the Free Source software PyTables Standard Edition. In brief, PyTables Pro is ready for getting the most of the hardware behind it, allowing to perform complex data analysis on datasets that are typically larger (and sometimes much larger) than your available memory. With typical usage, PyTables Pro can cope, while using similar resources, with datasets that can be up to 5x larger than what traditional databases would allow you.
PyTables Pro’s simple and elegant database design and operation capabilities reduce the need for database administrators. In addition, its outstanding test suite and years of accumulated experience, guarantee exceptionally high levels of quality and stability.
PyTables Pro is being actively used in a wide spectrum of fields, like aeronautics, drug discovery, financial analysis, telecommunications switching systems, data mining or statistical analysis, to name just a few, allowing to deal with extremely large datasets with maximum efficiency.
It comes with OPSI included
OPSI is a powerful and innovative indexing engine allowing PyTables Pro to perform really fast queries on arbitrarily large tables. Moreover, it offers a wide range of optimization levels for its indexes so that the user can choose the best one that suits her needs (more or less size, more or less performance). Indexation code also takes advantage of the vectorization capabilities of the NumPy and Numexpr packages to ensure really short indexing and search times.
By using OPSI it is possible to complete a query on a table in the order of ten thousand million (10,000,000,000) rows in a matter of hundredths of a second. Moreover, it can index columns in times between 1.5x and 25x shorter (depending on the table size and the desired optimization level) than other databases, while reducing the size of the indexes between 3x and 15x.
The main reason behind this incredible performance and compactness is that OPSI is geared towards tables that are mostly used for read-only or append-only purposes, and this is the scenario where it absolutely shines. If the user needs to update or delete frequently the values of indexed columns, then OPSI will take much more time than other solutions to keep its indexes updated. So, for situations that don't require fast updates or deletions, OPSI is probably one of the best indexing engines available.
Last but not least, and thanks to OPSI, PyTables Pro provides the ability to sort arbitrarily large tables by a specific field.
For more information about this innovative indexing engine, see the OPSI capabilities section below. And, although OPSI has got many improvements since this was written, you may also want to have a look at the OPSI White Paper, as many details exposed there are still valid for the current version.
Improved cache implementation
PyTables Pro wears a fine-tuned LRU cache for both metadata (nodes) and regular data that lets you achieve maximum speed for intensive object tree browsing and during data reads and queries. It complements the already efficient cache present in HDF5, although this is more geared towards high-level structures that are specific to PyTables Pro and that are critical for achieving very high performance.
Professional installer for Windows
No more need to chase across the web for missing pieces, or play the version numbers game. Both 32-bit and 64-bit flavors have been put at your disposal, so just download one single package, and you're ready to begin work. And, although all-in-one installers for other platforms are not available, PyTables Pro can be quickly deployed on these by using Python's distutils anyway.
Meant for production
Near 50,000 carefully designed test units (in 2.2 version) check every detail and feature of PyTables Pro. Besides, for every new version of the product, all the tests are verified to successfully pass for the most common platforms (Windows, Mac OS X, Linux, FreeBSD, all in both 32-bit and 64-bit flavors). This way, you can relax and concentrate your efforts in resolving your own problems (and not the underlying libraries).
Support from PyTables developers, support for PyTables developers
Every license purchased contributes to our development efforts, ensuring the future of the PyTables project. In addition, PyTables Pro site and developer licenses come with first-class, high priority support, that allows you to get help directly from PyTables developers and maximize your productivity.
Special capabilities implemented in OPSI
OPSI engine in PyTables Pro comes with a series of new and exciting features that cannot be found in other indexing engines. Among them you will find:
Four levels of index "lightness", namely ultralight, light, medium and full, are available so that the user will be able to choose the most appropriate for her needs. For example, ultralight indexes can take up to 4x less space than regular OPSI indexes, and up to more than 15x less space than indexes in traditional relational databases, while still providing an important query speed-up in many scenarios.
The index query engine is based on the concept of chunkmaps. This allows for a very effective way to retrieve table data in queries that have low selectivity (i.e. queries with a high number of rows as result), while retaining good performance for high selectivity ones.
A fine-tuned query optimizer being able to use several indexes simultaneously in a broad range of complex queries. This will greatly enhance the response times of potentially complicated queries.
Finally, OPSI permits to create completely sorted indexes (CSI), allowing not only to get better performance in queries, but also to sort arbitrarily large tables by a specific field. This can be very important if you are going to need top-performance in your queries.
With all these advanced features, consider what OPSI can do for accelerating the mining of your huge table, without wasting precious computer and man-time resources.