Anc*_*nia 3 ruby encoding web-crawler
可能重复:
ruby 1.9:UTF-8中的无效字节序列
我正在构建一个文件系统爬虫,并在运行我的脚本时收到以下错误:
wordcrawler.rb:8:in `block in <main>': invalid byte sequence in UTF-8 (ArgumentError)
from /Users/Anconia/.rvm/rubies/ruby-1.9.3-p327/lib/ruby/1.9.1/find.rb:41:in `block in find'
from /Users/Anconia/.rvm/rubies/ruby-1.9.3-p327/lib/ruby/1.9.1/find.rb:40:in `catch'
from /Users/Anconia/.rvm/rubies/ruby-1.9.3-p327/lib/ruby/1.9.1/find.rb:40:in `find'
from wordcrawler.rb:5:in `<main>'
Run Code Online (Sandbox Code Playgroud)
这是我的代码:
require 'find'
count = 0
Find.find('/Users/Anconia/') do |file| # '/' for root directory on OS X
if file =~ /\b(\.txt|\.doc|\.docx)\b/ # check if filename ends in desired format
contents = File.read(file)
if contents =~ /regex/
puts file
count += 1
end
end
end
puts "#{count} files were found"
Run Code Online (Sandbox Code Playgroud)
在我的开发环境中,我使用ruby 1.9.3; 但是,当我切换到ruby 1.8.7时,脚本运行正常.如果可能的话,我想继续使用1.9.3.我已经尝试过这篇文章中的每个解决方案(ruby 1.9:UTF-8中的无效字节序列),但我的问题仍然存在.有什么建议?
没有正确理解上述帖子的内容.至少这可以用作这篇文章的实现示例
require 'find'
count = 0
Find.find('/Users/Anconia/') do |file| # '/' for root directory on OS X
if file =~ /\b(\.txt|\.doc|\.docx)\b/ # check if filename ends in desired format
contents = File.read(file).encode!('UTF-8', 'UTF-8', :invalid => :replace) # resolves encoding errors - must use 1.9.3 else use iconv
if contents =~ /regex/
puts file
count += 1
end
end
end
puts "#{count} files were found"
Run Code Online (Sandbox Code Playgroud)