error trying to import from file

[paul.jobb]

Hi;

I am just evaluating the nagios log server and am trying to import a log file that I got for one of our isilon NAS appliances , I am using the following command

cat 00000438 | python shipper.py program:apache_access | nc 192.168.203.128 2057

and am getting the following, any thoughts would be appreciated

'utf8' codec can't decode byte 0xbe in position 0: invalid start byte
Traceback (most recent call last):
File "shipper.py", line 242, in <module>
main()
File "shipper.py", line 237, in main
process_stream(sys.stdin, message)
File "shipper.py", line 217, in process_stream
print json.dumps(message)
File "/usr/lib/python2.6/json/__init__.py", line 230, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.6/json/encoder.py", line 367, in encode
chunks = list(self.iterencode(o))
File "/usr/lib/python2.6/json/encoder.py", line 309, in _iterencode
for chunk in self._iterencode_dict(o, markers):
File "/usr/lib/python2.6/json/encoder.py", line 275, in _iterencode_dict
for chunk in self._iterencode(value, markers):
File "/usr/lib/python2.6/json/encoder.py", line 294, in _iterencode
yield encoder(o)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xbe in position 0: invalid start byte

[scottwilkerson]

Actually this looks like it could be a bug in shipper.py

Could I have you try making the following change in shipper.py ?

On line 216 change from

Code: Select all

        message['message'] = line

to

Code: Select all

        message['message'] = line.encode('utf-8')

And try running the command again.

If you continue to have an issue, would it be possible to have you PM me a couple lines from the file you are importing?

[paul.jobb]

Thanks Scott

I made the change and it gave me this

[root@localhost ~]# cat 00000438 | python shipper.py program:apache_access | nc 192.168.203.128 2057
'ascii' codec can't decode byte 0xbe in position 0: ordinal not in range(128)
Traceback (most recent call last):
File "shipper.py", line 243, in <module>
main()
File "shipper.py", line 238, in main
process_stream(sys.stdin, message)
File "shipper.py", line 217, in process_stream
message['message'] = line.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xbe in position 0: ordinal not in range(128)

Tomorrow I will take a look at that file and see whats in it and see if I can extract a portion of it for you.

[scottwilkerson]

Ya, we are going to have to take a look at the file it is trying to send...

Feel free to PM me a few lines and we can do some testing.