如果我有一个YEAR和一个WEEK数字,从中构造一个干净的方法是DATE什么?如果一周中的某天可以是星期一,我会更喜欢。
我试图从亚马逊使用scrapy获取数据我可以用CSV获取数据但我无法在mysql数据库中插入数据请找到我的代码 我的蜘蛛是
import scrapy
from craigslist_sample.items import AmazonDepartmentItem
from scrapy.contrib.spiders import CrawlSpider, Rule
from scrapy.contrib.linkextractors import LinkExtractor
class AmazonAllDepartmentSpider(scrapy.Spider):
name = "amazon"
allowed_domains = ["amazon.com"]
start_urls = [
"http://www.amazon.com/gp/site-directory/ref=nav_sad/187-3757581-3331414"
]
def parse(self, response):
for sel in response.xpath('//ul/li'):
item = AmazonDepartmentItem()
item['title'] = sel.xpath('a/text()').extract()
item['link'] = sel.xpath('a/@href').extract()
item['desc'] = sel.xpath('text()').extract()
return item
Run Code Online (Sandbox Code Playgroud)
我的管道代码是
import sys
import MySQLdb
import hashlib
from scrapy.exceptions import DropItem
from scrapy.http import Request
class MySQLStorePipeline(object):
host = 'rerhr.com'
user = 'amazon'
password = 'sads23'
db = 'amazon_project' …Run Code Online (Sandbox Code Playgroud)