Release notes for PyTables 2.0 series

Changes from 2.0b2 to 2.0rc1

  • The API Reference section of the User's Manual (and the matching docstrings) has been completely reviewed, expanded and corrected. This process has unveiled some errors and inconsistencies which have also been fixed.
  • Fixed VLArray.__getitem__() to behave as expected in Python when using slices, instead of following the semantics of PyTables' read() methods (e.g. reading just one element when no stop is provided). Fixes ticket #50.
  • Removed implicit UTF-8 encoding from VLArray data using vlstring atoms. Now a variable-length string is stored as is, which lets users use any encoding of their choice, or none of them. A vlunicode atom will probably be added to the next release so as to fix ticket #51.
  • Allow non-sequence objects to be passed to VLArray.append() when using an object atom. This was already possible in 1.x but stopped working when the old append syntax was dropped in 2.0. Fixes ticket #63.
  • Changed Cols.__len__() to return the number of rows of the table or nested column (instead of the number of fields), like its counterparts in Table and Column.
  • Python scalars cached in AttributeSet instances are now kept as NumPy objects instead of Python ones, because they do become NumPy objects when retrieved from disk. Fixes ticket #59.
  • Avoid HDF5 error when appending an empty array to a Table (ticket #57) or EArray (ticket #49) dataset.
  • Fix wrong implementation of the top-level table.description._v_dflts map, which was also including the pathnames of columns inside nested columns. Fixes ticket #45.
  • Optimized the access to unaligned arrays in Numexpr between a 30% and a 70%.
  • Fixed a die-hard bug that caused the loading of groups while closing a file. This only showed with certain usage patterns of the LRU cache (e.g. the one caused by ManyNodesTestCase in test_indexes.py under Pro).
  • Avoid copious warnings about unused functions and variables when compiling Numexpr.
  • Several fixes to Numexpr expressions with all constant values. Fixed tickets #53, #54, #55, #58. Reported bugs to mainstream developers.
  • Solved an issue when trying to open one of the included test files in append mode on a system-wide installation by a normal user with no write privileges on it. The file isn't being modified anyway, so the test is skipped then.
  • Added a new benchmark to compare the I/O speed of Array and EArray objects with that of cPickle.
  • The old Row.__call__() is no longer available as a public method. It was not documented, anyway. Fixes ticket #46.
  • Cols._f_close() is no longer public. Fixes ticket #47.
  • Attributes._f_close() is no longer public. Fixes ticket #52.
  • The undocumented Description.classdict attribute has been completely removed. Fixes ticket #44.

Changes from 2.0b1 to 2.0b2

  • A very exhaustive overhauling of the User's Manual is in process. The chapters 1 (Introduction), 2 (Installation), 3 (Tutorials) have been completed (and hopefully, the lines of code are easier to copy&paste now), while chapter 4 (API Reference) has been done up to (and including) the Table class. During this tedious (but critical in a library) overhauling work, we have tried hard to synchronize the text in the User's Guide with that which appears on the docstrings.
  • Removed the recursive argument in Group._f_walkNodes(). Using it with a false value was redundant with Group._f_iterNodes(). Fixes ticket #42.
  • Removed the coords argument from Table.read(). It was undocumented and redundant with Table.readCoordinates(). Fixes ticket #41.
  • Fixed the signature of Group.__iter__() (by removing its parameters).
  • Added new Table.coldescrs and Table.description._v_itemsize attributes.
  • Added a couple of new attributes for leaves:
    • nrowsinbuf: the number of rows that fit in the internal buffers.
    • chunkshape: the chunk size for chunked datasets.
  • Fixed setuptools so that making an egg out of the PyTables 2 package is possible now.
  • Added a new tables.restrict_flavors() function allowing to restrict available flavors to a given set. This can be useful e.g. if you want to force PyTables to get NumPy data out of an old, numarray-flavored PyTables file even if the numarray package is installed.
  • Fixed a bug which caused filters of unavailable compression libraries to be loaded as using the default Zlib library, after issuing a warning. Added a new FiltersWarning and a Filters.copy().

Changes from 1.4.x to 2.0b1

API additions

  • Column.createIndex() has received a couple of new parameters: optlevel and filters. The first one sets the desired quality level of the index, while the second one allows the user to specify the filters for the index.
  • Table.indexprops has been split into Table.indexFilters and Table.autoIndex. The later groups the functionality of the old auto and reindex.
  • The new Table.colpathnames is a sequence which contains the full pathnames of all bottom-level columns in a table. This can be used to walk all Column objects in a table when used with Table.colinstances.
  • The new Table.colinstances dictionary maps column pathnames to their associated Column or Cols object for simple or nested columns, respectively. This is similar to Table.cols._f_col(), but faster.
  • Row has received a new Row.fetch_all_fields() method in order to return all the fields in the current row. This returns a NumPy void scalar for each call.
  • New tables.test(verbose=False, heavy=False) high level function for interactively running the complete test suite from the Python console.
  • Added a tables.print_versions() for easily getting the versions for all the software on which PyTables relies on.

Backward-incompatible changes

  • You can no longer mark a column for indexing in a Col declaration. The only way of creating an index for a column is to invoke the createIndex() method of the proper column object after the table has been created.
  • Now the Table.colnames attribute is just a list of the names of top-level columns in a table. You can still get something similar to the old structure by using Table.description._v_nestedNames. See also the new Table.colpathnames attribute.
  • The File.objects, File.leaves and File.groups dictionaries have been removed. If you still need this functionality, please use the File.getNode() and File.walkNodes() instead.
  • Table.removeIndex() is no longer available; to remove an index on a column, one must use the removeIndex() method of the associated Column instance.
  • Column.dirty is no longer available. If you want to check column index dirtiness, use Column.index.dirty.
  • complib and complevel parameters have been removed from File.createTable(), File.createEArray(), File.createCArray() and File.createVLArray(). They were already deprecated in PyTables 1.x.
  • The shape and atom parameters have been swapped in File.createCArray(). This has been done to be consistent with Atom() definitions (i.e. type comes before and shape after).

Deprecated features

  • Node._v_rootgroup has been removed. Please use node._v_file.root instead.
  • The Node._f_isOpen() and Leaf.isOpen() methods have been removed. Please use the Node._v_isopen attribute instead (it is much faster).
  • The File.getAttrNode(), File.setAttrNode() and File.delAttrNode() methods have been removed. Please use File.getNodeAttr(), File.setNodeAttr() and File.delNodeAttr() instead.
  • File.copyAttrs() has been removed. Please use File.copyNodeAttrs() instead.
  • The table[colname] idiom is no longer supported. You can use table.cols._f_col(column) for doing the same.

API refinements

  • File.createEArray() received a new shape parameter. This allows to not have to use the shape of the atom so as to set the shape of the underlying dataset on disk.
  • All the leaf constructors have received a new chunkshape parameter that allows specifying the chunk sizes of datasets on disk.
  • All File.create*() factories for Leaf nodes have received a new byteorder parameter that allows the user to specify the byteorder in which data will be written to disk (data in memory is now always handled in native order).

Enjoy data!

—The PyTables Team

ReleaseNotes/Release 2.0rc1 (last edited 2008-04-21 11:12:45 by localhost)