[Erp5-dev] oood simple Excel files import problem

Discussion:

Łukasz Nowak

2008-05-05 09:38:32 UTC

Hello,

I'm using oood from revision 20841.

Attached file is .xls document made in OpenOffice.

OpenOffice version used is 2.3 (1:2.3.0-1ubuntu5.3).

While using python simple_cmdline_client.py OpenOfficeUx-2.1.0.xls
I've got such problem in oood.log:

2008/05/05 11:33:37 DEBUG <Thread-11936> we acquired semaphore
2008/05/05 11:33:37 DEBUG <Thread-11936> Thread-11938 is waiting for process to complete
2008/05/05 11:33:37 INFO <Thread-11936> [worker] trying to load /home/shufla/tmp/tmp/oood/0_OpenOfficeUx2.1.0.xls
2008/05/05 11:33:38 ERROR <Thread-11936> KeyError: 'MIMEType'
2008/05/05 11:33:38 ERROR <Thread-11936> File "/home/shufla/work/oood/worker.py", line 215, in _run
res = func(*args, **kwargs)
File "/home/shufla/work/oood/worker.py", line 149, in generate
return self.convert(kw)
File "/home/shufla/work/oood/worker.py", line 126, in convert
self._convert()
File "/home/shufla/work/oood/worker.py", line 316, in _convert
odf_format_info = mimemapper.getOdfInfo(self.metadata['MIMEType'])

2008/05/05 11:33:38 INFO <Thread-11936> [worker] resetting worker 0
2008/05/05 11:33:38 ERROR <Thread-11936> KeyError: 'MIMEType'
2008/05/05 11:33:38 ERROR <Thread-11936> File "/home/shufla/work/oood/dispatcher.py", line 234, in do_dispatch
data = self._process(worker, funcname, request)
File "/home/shufla/work/oood/dispatcher.py", line 397, in _process
res = method(kw) # res is not used, method puts processed data back into kw
File "/home/shufla/work/oood/worker.py", line 215, in _run
res = func(*args, **kwargs)
File "/home/shufla/work/oood/worker.py", line 149, in generate
return self.convert(kw)
File "/home/shufla/work/oood/worker.py", line 126, in convert
self._convert()
File "/home/shufla/work/oood/worker.py", line 316, in _convert
odf_format_info = mimemapper.getOdfInfo(self.metadata['MIMEType'])

2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Kill requested
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Is still running and span on several processes: 20552, 20568
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Killing process 20552...
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Killing process 20568...
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Killed
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Cleaned
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Starting...
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Listening at localhost:4060
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Starting waiting thread for process 21449
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Started as process 21449
2008/05/05 11:34:38 WARNING <Thread-11936> [factory] building worker for 0
2008/05/05 11:34:38 INFO <Thread-11936> [worker] resetting worker 0
2008/05/05 11:34:38 INFO <Thread-11936> [factory] worker for 0 created
2008/05/05 11:34:38 INFO <Thread-11936> [worker] trying to load /home/shufla/tmp/tmp/oood/0_OpenOfficeUx2.1.0.xls
2008/05/05 11:34:38 ERROR <Thread-11936> [worker] /home/shufla/tmp/tmp/oood/0_OpenOfficeUx2.1.0.xls could not be loaded into OpenOffice
2008/05/05 11:34:38 ERROR <Thread-11936> NotLoaded: /home/shufla/tmp/tmp/oood/0_OpenOfficeUx2.1.0.xls could not be loaded into OpenOffice
2008/05/05 11:34:38 ERROR <Thread-11936> File "/home/shufla/work/oood/worker.py", line 215, in _run
res = func(*args, **kwargs)
File "/home/shufla/work/oood/worker.py", line 149, in generate
return self.convert(kw)
File "/home/shufla/work/oood/worker.py", line 125, in convert
self._loadFile(kw['filename'], kw)
File "/home/shufla/work/oood/worker.py", line 260, in _loadFile
raise lib.NotLoaded(fname + ' could not be loaded into OpenOffice')

2008/05/05 11:34:38 INFO <Thread-11936> [worker] resetting worker 0
2008/05/05 11:34:38 ERROR <Thread-11936> NotLoaded: /home/shufla/tmp/tmp/oood/0_OpenOfficeUx2.1.0.xls could not be loaded into OpenOffice
2008/05/05 11:34:38 ERROR <Thread-11936> File "/home/shufla/work/oood/dispatcher.py", line 246, in do_dispatch
data = self._process(worker, funcname, request)
File "/home/shufla/work/oood/dispatcher.py", line 422, in _process
return method(request)
File "/home/shufla/work/oood/worker.py", line 215, in _run
res = func(*args, **kwargs)
File "/home/shufla/work/oood/worker.py", line 149, in generate
return self.convert(kw)
File "/home/shufla/work/oood/worker.py", line 125, in convert
self._loadFile(kw['filename'], kw)
File "/home/shufla/work/oood/worker.py", line 260, in _loadFile
raise lib.NotLoaded(fname + ' could not be loaded into OpenOffice')

2008/05/05 11:34:38 INFO <Thread-11936> 402

Regards,
Luke
--
?ukasz Nowak R&D Ventis http://www.ventis.com.pl/
tel: +48 32 768 16 85 fax: +48 32 392 10 61
``Use the Source, Luke...'' I am only craftsman.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenOfficeUx-2.1.0.xls
Type: application/vnd.ms-excel
Size: 63488 bytes
Desc: not available
URL: <http://mail.tiolive.com/pipermail/erp5-dev/attachments/20080505/9c2de883/attachment.xls>

bartek

2008-05-05 09:46:32 UTC

Permalink

Post by Åukasz Nowak
Hello,
I'm using oood from revision 20841.
Attached file is .xls document made in OpenOffice.
OpenOffice version used is 2.3 (1:2.3.0-1ubuntu5.3).
While using python simple_cmdline_client.py OpenOfficeUx-2.1.0.xls

Thanks - I'll check it later today. I think it the problem is in the
client, not in the oood.

Bartek

Post by Åukasz Nowak
2008/05/05 11:33:37 DEBUG <Thread-11936> we acquired semaphore
2008/05/05 11:33:37 DEBUG <Thread-11936> Thread-11938 is waiting for process to complete
2008/05/05 11:33:37 INFO <Thread-11936> [worker] trying to load /home/shufla/tmp/tmp/oood/0_OpenOfficeUx2.1.0.xls
2008/05/05 11:33:38 ERROR <Thread-11936> KeyError: 'MIMEType'
2008/05/05 11:33:38 ERROR <Thread-11936> File "/home/shufla/work/oood/worker.py", line 215, in _run
res = func(*args, **kwargs)
File "/home/shufla/work/oood/worker.py", line 149, in generate
return self.convert(kw)
File "/home/shufla/work/oood/worker.py", line 126, in convert
self._convert()
File "/home/shufla/work/oood/worker.py", line 316, in _convert
odf_format_info = mimemapper.getOdfInfo(self.metadata['MIMEType'])
2008/05/05 11:33:38 INFO <Thread-11936> [worker] resetting worker 0
2008/05/05 11:33:38 ERROR <Thread-11936> KeyError: 'MIMEType'
2008/05/05 11:33:38 ERROR <Thread-11936> File "/home/shufla/work/oood/dispatcher.py", line 234, in do_dispatch
data = self._process(worker, funcname, request)
File "/home/shufla/work/oood/dispatcher.py", line 397, in _process
res = method(kw) # res is not used, method puts processed data back into kw
File "/home/shufla/work/oood/worker.py", line 215, in _run
res = func(*args, **kwargs)
File "/home/shufla/work/oood/worker.py", line 149, in generate
return self.convert(kw)
File "/home/shufla/work/oood/worker.py", line 126, in convert
self._convert()
File "/home/shufla/work/oood/worker.py", line 316, in _convert
odf_format_info = mimemapper.getOdfInfo(self.metadata['MIMEType'])
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Kill requested
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Is still running and span on several processes: 20552, 20568
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Killing process 20552...
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Killing process 20568...
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Killed
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Cleaned
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Starting...
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Listening at localhost:4060
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Starting waiting thread for process 21449
2008/05/05 11:33:38 INFO <Thread-11936> Instance #0 - Started as process 21449
2008/05/05 11:34:38 WARNING <Thread-11936> [factory] building worker for 0
2008/05/05 11:34:38 INFO <Thread-11936> [worker] resetting worker 0
2008/05/05 11:34:38 INFO <Thread-11936> [factory] worker for 0 created
2008/05/05 11:34:38 INFO <Thread-11936> [worker] trying to load /home/shufla/tmp/tmp/oood/0_OpenOfficeUx2.1.0.xls
2008/05/05 11:34:38 ERROR <Thread-11936> [worker] /home/shufla/tmp/tmp/oood/0_OpenOfficeUx2.1.0.xls could not be loaded into OpenOffice
2008/05/05 11:34:38 ERROR <Thread-11936> NotLoaded: /home/shufla/tmp/tmp/oood/0_OpenOfficeUx2.1.0.xls could not be loaded into OpenOffice
2008/05/05 11:34:38 ERROR <Thread-11936> File "/home/shufla/work/oood/worker.py", line 215, in _run
res = func(*args, **kwargs)
File "/home/shufla/work/oood/worker.py", line 149, in generate
return self.convert(kw)
File "/home/shufla/work/oood/worker.py", line 125, in convert
self._loadFile(kw['filename'], kw)
File "/home/shufla/work/oood/worker.py", line 260, in _loadFile
raise lib.NotLoaded(fname + ' could not be loaded into OpenOffice')
2008/05/05 11:34:38 INFO <Thread-11936> [worker] resetting worker 0
2008/05/05 11:34:38 ERROR <Thread-11936> NotLoaded: /home/shufla/tmp/tmp/oood/0_OpenOfficeUx2.1.0.xls could not be loaded into OpenOffice
2008/05/05 11:34:38 ERROR <Thread-11936> File "/home/shufla/work/oood/dispatcher.py", line 246, in do_dispatch
data = self._process(worker, funcname, request)
File "/home/shufla/work/oood/dispatcher.py", line 422, in _process
return method(request)
File "/home/shufla/work/oood/worker.py", line 215, in _run
res = func(*args, **kwargs)
File "/home/shufla/work/oood/worker.py", line 149, in generate
return self.convert(kw)
File "/home/shufla/work/oood/worker.py", line 125, in convert
self._loadFile(kw['filename'], kw)
File "/home/shufla/work/oood/worker.py", line 260, in _loadFile
raise lib.NotLoaded(fname + ' could not be loaded into OpenOffice')
2008/05/05 11:34:38 INFO <Thread-11936> 402
Regards,
Luke
------------------------------------------------------------------------
_______________________________________________
Erp5-dev mailing list
Erp5-dev at erp5.org
http://mail.nexedi.com/mailman/listinfo/erp5-dev

--
"feelings affect productivity. (...) unhappy people write worse
software, and less of it."
Karl Fogel, "Producing Open Source Software"

Łukasz Nowak

2008-05-05 09:49:57 UTC

Permalink

On 2008-05-05, 11:46:32

Post by bartek

Thanks - I'll check it later today. I think it the problem is in the
client, not in the oood.

Hello - same problem is while doing this in ERP5 (forgot to mention
it). I just wanted to be repetitive, so that's why I used python client
in this case.

Regards,
Luke

--
?ukasz Nowak R&D Ventis http://www.ventis.com.pl/
tel: +48 32 768 16 85 fax: +48 32 392 10 61
``Use the Source, Luke...'' I am only craftsman.

Łukasz Nowak

2008-05-05 09:50:50 UTC

Permalink

On 2008-05-05, 11:49:57

Post by Åukasz Nowak
On 2008-05-05, 11:46:32

Post by bartek

Thanks - I'll check it later today. I think it the problem is in
the client, not in the oood.

Hello - same problem is while doing this in ERP5 (forgot to mention
it). I just wanted to be repetitive, so that's why I used python
client in this case.

ERP5's traceback:

2008-05-05T11:32:52 ERROR Zope.SiteErrorLog http://:
Traceback (most recent call last):
File "/home/shufla/work/erp5head/unit_test/Products/CMFActivity/ActivityTool.py", line 228, in __call__
result = method(*self.args, **self.kw)
File "/usr/lib/erp5/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__
return self._bindAndExec(args, kw, None)
File "/usr/lib/erp5/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec
return self._exec(bound_data, args, kw)
File "/usr/lib/erp5/lib/python/Products/PythonScripts/PythonScript.py", line 324, in _exec
result = f(*args, **kw)
File "Script (Python)", line 9, in Document_convertToBaseFormatAndDiscoverMetadata
File "/home/shufla/work/erp5head/unit_test/Products/ERP5/Document/Document.py", line 1277, in convertToBaseFormat
message = self._convertToBaseFormat() # Call implemetation method
File "/home/shufla/work/erp5head/unit_test/Products/ERP5OOo/Document/OOoDocument.py", line 527, in _convertToBaseFormat
raise ConversionError(
ConversionError: OOoDocument: Error converting document to base format 402::

Regards,
Luke

--
?ukasz Nowak R&D Ventis http://www.ventis.com.pl/
tel: +48 32 768 16 85 fax: +48 32 392 10 61
``Use the Source, Luke...'' I am only craftsman.

Łukasz Nowak

2008-05-05 09:52:44 UTC

Permalink

Hello,

On 2008-05-05, 11:49:57

Post by Åukasz Nowak
On 2008-05-05, 11:46:32

Post by bartek

Thanks - I'll check it later today. I think it the problem is in
the client, not in the oood.

Hello - same problem is while doing this in ERP5 (forgot to mention
it). I just wanted to be repetitive, so that's why I used python
client in this case.

And again - this problem raises, while trying convert to "OpenDocument
Spreadsheet" only.

Regards,
Luke

--
?ukasz Nowak R&D Ventis http://www.ventis.com.pl/
tel: +48 32 768 16 85 fax: +48 32 392 10 61
``Use the Source, Luke...'' I am only craftsman.

Łukasz Nowak

2008-05-06 10:43:27 UTC

Permalink

Hello,

On 2008-05-05, 11:46:32

Post by bartek

Thanks - I'll check it later today. I think it the problem is in the
client, not in the oood.

This problem is not appearing on clean Mandriva Linux release 2008.1
(Official) for i586 with openoffice.org-oood-0.3.1-4nxd2008.1

I'll try to figure out what's wrong.

Regards,
Luke

--
?ukasz Nowak R&D Ventis http://www.ventis.com.pl/
tel: +48 32 768 16 85 fax: +48 32 392 10 61
``Use the Source, Luke...'' I am only craftsman.

Łukasz Nowak

2008-05-06 10:53:17 UTC

Permalink

Hello,

On 2008-05-06, 12:43:27

Post by Åukasz Nowak
Hello,
On 2008-05-05, 11:46:32

Post by bartek

Thanks - I'll check it later today. I think it the problem is in
the client, not in the oood.

This problem is not appearing on clean Mandriva Linux release 2008.1
(Official) for i586 with openoffice.org-oood-0.3.1-4nxd2008.1
I'll try to figure out what's wrong.

Problem reappeared on current trunk revision on mandriva system.

How can I figure out which revision was used to built those rpm?

Regards,
Luke

--
?ukasz Nowak R&D Ventis http://www.ventis.com.pl/
tel: +48 32 768 16 85 fax: +48 32 392 10 61
``Use the Source, Luke...'' I am only craftsman.

Jérome Perrin

2008-05-20 08:20:21 UTC

Permalink

Post by Åukasz Nowak
Hello,
On 2008-05-05, 11:46:32

Post by bartek

Thanks - I'll check it later today. I think it the problem is in the
client, not in the oood.

This problem is not appearing on clean Mandriva Linux release 2008.1
(Official) for i586 with openoffice.org-oood-0.3.1-4nxd2008.1
I'll try to figure out what's wrong.

I really suspect that the problem is with the openoffice version used.

I think the latest OOo version is selected automatically when installing
oood.

I tried oood with different openoffice versions:

Openoffice 2.4.0:
http://mail.nexedi.com/pipermail/erp5-report/2008-May/021813.html

Openoffice 2.3.1:
http://mail.nexedi.com/pipermail/erp5-report/2008-May/021844.html

2.3.1 seems to have problems converting spreadsheets. 2.4.0 works
better, but have problems converting text.
The "ERP5" test runner uses old oood with openoffice 2.1 which works
reasonably good.

J?rome

Łukasz Nowak

2008-05-20 09:03:48 UTC

Permalink

Hello,

On 2008-05-20, 10:20:21
J?rome Perrin <jerome at nexedi.com> wrote:

(...)

Post by JÃ©rome Perrin
I really suspect that the problem is with the openoffice version used.
I think the latest OOo version is selected automatically when
installing oood.
http://mail.nexedi.com/pipermail/erp5-report/2008-May/021813.html
http://mail.nexedi.com/pipermail/erp5-report/2008-May/021844.html
2.3.1 seems to have problems converting spreadsheets. 2.4.0 works
better, but have problems converting text.
The "ERP5" test runner uses old oood with openoffice 2.1 which works
reasonably good.

By converting text do you mean simple text files?

Well - so as a summary - 2.1 version is the best one?

Regards,
Luke

--
?ukasz Nowak R&D Ventis http://www.ventis.com.pl/
tel: +48 32 768 16 85 fax: +48 32 392 10 61
``Use the Source, Luke...'' I am only craftsman.

Jérome Perrin

2008-05-20 11:07:13 UTC

Permalink

Post by Åukasz Nowak
Hello,
On 2008-05-20, 10:20:21
(...)

used.

Post by JÃ©rome Perrin
I think the latest OOo version is selected automatically when
installing oood.
http://mail.nexedi.com/pipermail/erp5-report/2008-May/021813.html
http://mail.nexedi.com/pipermail/erp5-report/2008-May/021844.html
2.3.1 seems to have problems converting spreadsheets. 2.4.0 works
better, but have problems converting text.
The "ERP5" test runner uses old oood with openoffice 2.1 which works
reasonably good.

By converting text do you mean simple text files?

No, I mean .odt or .doc ; test_05_FormatIngestionText
(testIngestion.TestIngestion) is failing with 2.4.0, but it's the only
failure reported by ERP5 test suite.

Post by Åukasz Nowak
Well - so as a summary - 2.1 version is the best one?

ERP5 test suite also have a few failures with 2.1, and 2.4 is also not
bad for our use.

I just wanted to point out that we have different results with different
openoffice versions, but I can't say which version is the best one. I
myself use 2.4 now, because it's the latest and most tests pass.

J?rome

Łukasz Nowak

2008-05-20 11:13:13 UTC

Permalink

Hello,

On 2008-05-20, 13:07:13
J?rome Perrin <jerome at nexedi.com> wrote:

(...)

Post by Åukasz Nowak
By converting text do you mean simple text files?

No, I mean .odt or .doc ; test_05_FormatIngestionText
(testIngestion.TestIngestion) is failing with 2.4.0, but it's the only
failure reported by ERP5 test suite.

Ok. Understood. But simple .txt are supported anyway?

Post by Åukasz Nowak
Well - so as a summary - 2.1 version is the best one?

ERP5 test suite also have a few failures with 2.1, and 2.4 is also not
bad for our use.
I just wanted to point out that we have different results with
different openoffice versions, but I can't say which version is the
best one. I myself use 2.4 now, because it's the latest and most
tests pass.

And as it is used in mdv2008.1 I just use 2.4 version.

Thanks for support.
Luke

--
?ukasz Nowak R&D Ventis http://www.ventis.com.pl/
tel: +48 32 768 16 85 fax: +48 32 392 10 61
``Use the Source, Luke...'' I am only craftsman.