V1 file format

Feather was created early in the Arrow project as a proof of concept for fast, language-agnostic data frame storage for Python pandas and R.

There are two file format versions for Feather:. Version 2 V2 , the default version, which is exactly represented as the Arrow IPC file format on disk. V2 was first made available in Apache Arrow 0. Version 1 V1 , a legacy version available starting in , replaced by V2. V1 files are distinct from Arrow IPC files and lack many features, such as the ability to store all Arrow data types.

V1 files also lack compression support. We intend to maintain read support for V1 for the foreseeable future. The pyarrow. DataFrame object:.

As of Apache Arrow version 0. LZ4 is used by default if it is available which it should be if you obtained pyarrow through a normal package manager :. Note that the default LZ4 compression generally yields much smaller files without sacrificing much read or write performance. In some instances, LZ4-compressed files may be faster to read and write than uncompressed due to reduced disk IO requirements. DataType pyarrow. DictionaryType pyarrow.

ListType pyarrow. MapType pyarrow. StructType pyarrow. UnionType pyarrow. TimestampType pyarrow. Time32Type pyarrow. Time64Type pyarrow. FixedSizeBinaryType pyarrow. DecimalType pyarrow. Field pyarrow.

Schema pyarrow. ExtensionType pyarrow. PyExtensionType pyarrow. Array pyarrow. BooleanArray pyarrow. FloatingPointArray pyarrow. IntegerArray pyarrow. Int8Array pyarrow. Int16Array pyarrow. Int32Array pyarrow. Int64Array pyarrow. NullArray pyarrow. NumericArray pyarrow. UInt8Array pyarrow. UInt16Array pyarrow.

UInt32Array pyarrow. UInt64Array pyarrow. BinaryArray pyarrow. StringArray pyarrow. FixedSizeBinaryArray pyarrow. LargeBinaryArray pyarrow. LargeStringArray pyarrow. Time32Array pyarrow. Time64Array pyarrow. Date32Array pyarrow. Date64Array pyarrow. TimestampArray pyarrow.

DurationArray pyarrow. MonthDayNanoIntervalArray pyarrow. DecimalArray pyarrow. DictionaryArray pyarrow. ListArray pyarrow. FixedSizeListArray pyarrow. LargeListArray pyarrow. StructArray pyarrow.

UnionArray pyarrow. ExtensionArray pyarrow. If you know one, please use the 'Suggest a program' link below. Try a universal file viewer. Since we do not have any programs listed that we have verified can open V1 files, we suggest that you try a universal file viewer like Free File Viewer.

It can open over different types of files - and very likely yours too! Download Free File Viewer. Not sure exactly what type of file you are trying to open? Try our new File Analyzer. It is a free tool that can identify more than 11, different kinds of files - most likely yours too! I'm a Reporter. Staff Profiles. Social Media. Contact Us. About Us. Survey Manual. Key Officials. Careers and Employees. Doing Business. Emergency Management. File Formats By Data Management. File formats are standard methods for encoding digital information.

File Format Examples. Table of Contents. Best practices for choosing a file format for acquisition Collect data in a file format that is open and non-proprietary to limit the need to convert from one format to another If you need to collect data in a proprietary format, ensure that it can easily be converted to another non-proprietary, open format. Select formats that have broad use and support in your community Be aware of software, hardware, and licensing requirements for viewing and working with the data When possible, choose formats that are self-describing and can automatically capture metadata Within the files, avoid application of formatting, such as highlighting or color, to serve as metadata, because it will likely be lost when converting to different formats Best practices for public data release formats Data for public release must be in open, non-proprietary, and machine-readable formats Release data in multiple formats if the format used by the scientific community does not meet all of these requirements Due to file transfer limitations, compress data using lossless formats If sharing different formats of the same file, be sure to name each file with the same name e.

Check the actual data: column headings, rows, etc Ensure that values were not truncated and that significant figures were preserved Check the metadata; make sure it is present and accurate. Best practices for long-term preservation Save data in open, non-proprietary, unencrypted formats for long-term preservation Proprietary formats used for acquisition and analysis should be converted into standard and long lasting formats by the researcher familiar with the data, once the data analysis is complete.

Check the actual data itself: column headings, rows, etc Ensure that values were not truncated and that significant figures were preserved Check the internal metadata; make sure it is present and accurate. Save data in uncompressed formats whenever possible If saving an image file using compression, lossless is better than lossy. Australian National Data Service. File Formats. Sustainable Data Formats.