Page 1 of 1

error trying to import from file

Posted: Fri Oct 24, 2014 4:30 pm
by paul.jobb
Hi;

I am just evaluating the nagios log server and am trying to import a log file that I got for one of our isilon NAS appliances , I am using the following command

cat 00000438 | python shipper.py program:apache_access | nc 192.168.203.128 2057

and am getting the following, any thoughts would be appreciated

'utf8' codec can't decode byte 0xbe in position 0: invalid start byte
Traceback (most recent call last):
File "shipper.py", line 242, in <module>
main()
File "shipper.py", line 237, in main
process_stream(sys.stdin, message)
File "shipper.py", line 217, in process_stream
print json.dumps(message)
File "/usr/lib/python2.6/json/__init__.py", line 230, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.6/json/encoder.py", line 367, in encode
chunks = list(self.iterencode(o))
File "/usr/lib/python2.6/json/encoder.py", line 309, in _iterencode
for chunk in self._iterencode_dict(o, markers):
File "/usr/lib/python2.6/json/encoder.py", line 275, in _iterencode_dict
for chunk in self._iterencode(value, markers):
File "/usr/lib/python2.6/json/encoder.py", line 294, in _iterencode
yield encoder(o)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xbe in position 0: invalid start byte

Re: error trying to import from file

Posted: Fri Oct 24, 2014 5:28 pm
by scottwilkerson
Actually this looks like it could be a bug in shipper.py

Could I have you try making the following change in shipper.py ?

On line 216 change from

Code: Select all

        message['message'] = line
to

Code: Select all

        message['message'] = line.encode('utf-8')
And try running the command again.

If you continue to have an issue, would it be possible to have you PM me a couple lines from the file you are importing?

Re: error trying to import from file

Posted: Mon Oct 27, 2014 5:08 pm
by paul.jobb
Thanks Scott

I made the change and it gave me this

[root@localhost ~]# cat 00000438 | python shipper.py program:apache_access | nc 192.168.203.128 2057
'ascii' codec can't decode byte 0xbe in position 0: ordinal not in range(128)
Traceback (most recent call last):
File "shipper.py", line 243, in <module>
main()
File "shipper.py", line 238, in main
process_stream(sys.stdin, message)
File "shipper.py", line 217, in process_stream
message['message'] = line.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xbe in position 0: ordinal not in range(128)

Tomorrow I will take a look at that file and see whats in it and see if I can extract a portion of it for you.

Re: error trying to import from file

Posted: Tue Oct 28, 2014 9:21 am
by scottwilkerson
Ya, we are going to have to take a look at the file it is trying to send...

Feel free to PM me a few lines and we can do some testing.