0

我在网上抓取这个网址:http ://www.rajtamil.com/category/vijay-tv-shows/

遇到此错误:

    movTitle = str(link['title'])
    UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 41: ordinal not in range(128)

这是我的代码片段

    rajTamilurl='http://www.rajtamil.com/category/vijay-tv-shows/'
    req = urllib2.Request(rajTamilurl)
    req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3')
    response = urllib2.urlopen(req)
    link=response.read()
    response.close()

    #Here's what i've tried so far
    #link=link.decode('utf-8')
    #link=link.encode('utf-8','ignore')
    #link=link.decode('ascii', 'ignore')
    #soup = BeautifulSoup(link, from_encoding="utf-8")
    #soup = BeautifulSoup(link.decode('utf-8','ignore'))
    #soup = BeautifulSoup(link, 'html5lib')
    #print soup.prettify()

    soup = BeautifulSoup(link)
    for eachItem in soup.findAll('li'):
        for coveritem in eachItem.findAll("div", { "class":"cover" }):
            links = coveritem.find_all('a')
            for link in links:
                print link['title']
                movTitle = str(link['title'])

任何指针?

4

1 回答 1

2

虽然我强烈不建议使用str,但我了解您有一些限制。尝试更改此行:

movTitle = str(link['title'])

有了这个:

movTitle = link['title'].encode('utf8')

当你encode是一个unicode字符串时,你会得到它各自的编码str版本。

希望这可以帮助!

于 2014-02-21T22:51:44.690 回答