ruby+nokogori抓取糗事百科前10页并存储进数据库示例

2020-03-29 12:10:57

字体：大中小

来源：转载

供稿：网友

ruby,nokogori,爬取糗事百科最新的10页加图片比并同时保存进文本跟数据库

复制代码代码如下:

	
	#encoding:utf-8
	require "open-uri"
	require "nokogiri"
	require "mysql"
	@dbh=Mysql.real_connect("localhost","root","dengli","pachong")
	@file=open("qb.txt","w")
	def getQiubai(url)
	  data=open(url){|f| f.read}
	  doc=Nokogiri::HTML(data)
	  doc.css('div.block.untagged.mb15.bs2').each{|p|
	    content=p.css('div.content').text
	    @file.puts("#{content}/n")
	    img=p.css('div.thumb img').each{|img|
	       imgAddr=img.attr('src')
	       @file.puts("#{imgAddr}/n")
	       time=Time.now
	       sql="INSERT INTO qiushibaike (`body`,`img`,`time`) VALUES ('#{content}','#{imgAddr}','#{time}')"
	       @dbh.query(sql)
	       }
	  }
	end
	for i in 1..10
	  @file.puts("这是第#{i}页")
	  url="http://www.qiushibaike.com/8hr/page/#{i}"
	  getQiubai(url)
	end