Issue 123340 - Feature Request: Increase limits (maximum number of rows and cells) /Unable to import large text-tab delimited document
Summary: Feature Request: Increase limits (maximum number of rows and cells) /Unable t...
Status: CONFIRMED
Alias: None
Product: Calc
Classification: Application
Component: open-import (show other issues)
Version: 4.0.0
Hardware: All All
: P3 Normal (vote)
Target Milestone: ---
Assignee: AOO issues mailing list
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-09-24 14:10 UTC by webassistant5.tft+openoffice
Modified: 2013-10-21 15:28 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description webassistant5.tft+openoffice 2013-09-24 14:10:53 UTC
I am unable to import a large text-tab delimited document with the following specifications:
---------------
Rows: 60,249

Columns: 18

Cells: 1,020,126

Maximum characters per cell: 268

--------------
The rows are imported but cropped at 60,226 rows (-23 from the actual document). And if I try LibreOffice fork of OpenOffice I also get the error "The data could not be loaded completely because the maximum number of characters per cell was exceeded." and nothing imports at all. At least OpenOffice is able to open the document, but not completely.

Feature Request:
As I see it the Microsoft Excel specs are much higher than OpenOffice currently, even the older 2010 version. Work should be done to allow OpenOffice to work with Big(er) Data and large documents.
https://office.microsoft.com/en-us/excel-help/excel-specifications-and-limits-HP010073849.aspx
Comment 1 Edwin Sharp 2013-09-25 17:59:08 UTC
Can you please share the text document?
Comment 2 webassistant5.tft+openoffice 2013-10-04 16:03:46 UTC
(In reply to Edwin Sharp from comment #1)
> Can you please share the text document?

As the data contains inventory information I am not able to share the exact document. I have been able to re-create a similar document with Lorem Ipsum however; which is  downloadable here: 
http://www.fileconvoy.com/dfl.php?id=gb8fb948b65603142999385510b7c2a36510c78e6a

This document gets input/output errors in Libre Office and locks up the latest OpenOffice, but works perfectly in Microsoft Excel. Also it doesn't matter whether it is text-tab delimited/xls/etc.
Comment 3 Edwin Sharp 2013-10-06 04:57:45 UTC
Thank you.
File opens OK in Excel 2007.
Calc Rev. 1525015 not responding.
Win 7
Comment 4 Prosper Uniger 2013-10-06 16:57:38 UTC
The document from comment 2 takes ages to open because it got some very long texts in some cells,
7208 characters.

A xslx-file with one 7208 character long text cell takes half a minute on my machine, now multiply with several thousand fields. 
(maybe file new performance bug for this)

------------

please provide a new sample with shorter text in cells to avoid the performance bug.

My Excel 2003 caps the lines at 65536 while your file has 70000 lines.
I reduced the text to 268 characters like stated in comment 1 and saved as Excel 2007 xlsx
-> AOO 4.0.1 and 3.4.1 open the new file just fine, all 65536 lines.
Comment 5 webassistant5.tft+openoffice 2013-10-21 15:28:34 UTC
(In reply to Prosper Uniger from comment #4)
> The document from comment 2 takes ages to open because it got some very long
> texts in some cells,
> 7208 characters.
> 
> A xslx-file with one 7208 character long text cell takes half a minute on my
> machine, now multiply with several thousand fields. 
> (maybe file new performance bug for this)
> 
> ------------
> 
> please provide a new sample with shorter text in cells to avoid the
> performance bug.
> 
> My Excel 2003 caps the lines at 65536 while your file has 70000 lines.
> I reduced the text to 268 characters like stated in comment 1 and saved as
> Excel 2007 xlsx
> -> AOO 4.0.1 and 3.4.1 open the new file just fine, all 65536 lines.

Seems to be an issue with cells like you said. 

I've opened the new bug here per your advice:
https://issues.apache.org/ooo/show_bug.cgi?id=123511

However if one were using only OpenOffice it would be difficult to trim the lines down before importing. And even if this were automatic it might loose valuable description data.