Bootstrap FreeKB - Python (Scripting) - Resolve UnicodeDecodeError
Python (Scripting) - Resolve UnicodeDecodeError

Updated:   |  Python (Scripting) articles

Let's say your Python script returns something like this.

Traceback (most recent call last):
  File "example.py", line 10, in <module>
    logging.info(item.decode('utf-8'))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x83 in position 79: invalid start byte

 

I often happen upon this when using decode to convert a bytes object into string. For example, let's say you have a list that contains byte objects.

for item in items:
  print(item)

 

You will know the objects are bytes as each element will begin with b. Also notice in this example that line 3 contains \x83. This is what is causing the UnicodeDecodeError exception to be raised.

b'VMware\'s Customer Experience Improvement Program ("CEIP") provides VMware with information that enables VMware to '
b'improve its products and services, to fix problems, and to advise you on how best to deploy and use our products.  As '
b'part of the CEIP, VMware collects technical information about your organization\x83?Ts use of VMware products and services'

 

decode can be used to convert the byte objects into strings.

for item in items:
  print(item.decode('utf-8'))

 

A try / except / else block can be used.

for item in items:
  try:
    item.decode('utf-8')
  except UnicodeDecodeError as exception:
    print(f"UnicodeDecodeError exception raised: {exception}")
  else:
    print(item.decode('utf-8'))

 

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


Add a Comment


Please enter 1ba149 in the box below so that we can be sure you are a human.