美文网首页
EX05 Amazon 的Robots协议代码解释

EX05 Amazon 的Robots协议代码解释

作者: 胡云云云云 | 来源:发表于2018-05-10 21:10 被阅读0次

    以下是对Amazon网站的Robots协议的部分解释,很多文件夹的命名看不懂意思,大致说明了一下。

    User-agent: *    //*是一个通配符,表示所有的搜索引擎种类,这里亚马逊禁止所有的搜索引擎爬取以下目录及目录下的文件

    Disallow: /buycar

    Disallow: /cart

    Disallow: /checkout

    Disallow: /class

    Disallow: /com

    Disallow: /common

    Disallow: /css

    Disallow: /dll

    Disallow: /doc    //这些根目录


     //屏蔽dp和gp文件目录下的大部分文件目录,dp主要是针对用户信息建立的,gp主要是针对商品信息建立的,包括

    Disallow: /dp/e-mail-friend/    //好友邮件地址(大概吧)

    Disallow: /dp/manual-submit/   

    Disallow: /dp/product-availability/   //产品有效性

    Disallow: /dp/rate-this-item/    //项目的rate

    Disallow: /dp/shipping/    //物流信息

    Disallow: /dp/twister-update/

    Disallow: /gp/aws/ssop

    Disallow: /gp/cart    //购物车信息

    Disallow: /gp/css/homepage.html    //gp目录下的css目录下的HTML文件

    Disallow: /gp/customer-reviews/common/du    //顾客浏览纪录

    Disallow: /gp/flex

    Disallow: /gp/gfix

    Disallow: /gp/history    //历史记录

    Disallow: /gp/item-dispatch

    Disallow: /gp/music/clipserve

    Disallow: /gp/music/wma-pop-up

    Disallow: /gp/offer-listing    //供应商列表

    Disallow: /gp/product/e-mail-friend

    Disallow: /gp/product/product-availability

    Disallow: /gp/product/rate-this-item

    Disallow: /gp/recsradio

    Disallow: /gp/slredirect

    Disallow: /gp/twitter/

    Disallow: /gp/vote    //商品投票之类的

    Disallow: /gp/voting/

    Disallow: /gp/yourstore   


    //还屏蔽了一些配置文件

    Disallow: /inc

    Disallow: /js

    Disallow: /lib


    //mn目录文件下的一些子目录,mn可能是为亚马逊旗下提供的一些功能所建立的文件目录

    Disallow: /mn/bookLookInsideApp    

    Disallow: /mn/checkInitApp

    Disallow: /mn/checkoutAlertMsgApp

    Disallow: /mn/checkoutredirectApp

    Disallow: /mn/giftCardApp    //礼物卡

    Disallow: /mn/loginApplication    //登录申请

    Disallow: /mn/loyaltyApp    //会员服务

    Disallow: /mn/orderAddrApp

    Disallow: /mn/orderCfmApp

    Disallow: /mn/orderDetailApp

    Disallow: /mn/orderFailApp

    Disallow: /mn/orderHistoryApp

    Disallow: /mn/orderModifyApp

    Disallow: /mn/orderSummaryApp

    Disallow: /mn/paymentRedriveApp

    Disallow: /mn/recommendReviewApp

    Disallow: /mn/releaseReviewApp

    Disallow: /mn/reviewVoteApplication

    Disallow: /mn/selectPaymentMethodApp

    Disallow: /mn/selectShippingOpptionApplication

    Disallow: /mn/shipmentTraceApp

    Disallow: /mn/shoppingCartApplication

    Disallow: /mn/tellFriend

    Disallow: /mn/thankYouApplication

    Disallow: /mn/virtualAccountApp

    Disallow: /mn/yourAccountApp


    Disallow: /paper

    Disallow: /xml

    Disallow: /youraccount

    Disallow: /ap/signin    //个人登录信息

    Disallow: /gp/registry/wishlist/    /商品愿望清单


    Disallow: /wishlist/    //这里,Amazon虽然屏蔽了用户愿望清单文件目录,但是对于其下的几个开放了权限,包括公开的愿望清单等

    Allow: /wishlist/universal*

    Allow: /wishlist/vendor-button*

    Allow: /wishlist/get-button*


    Disallow: /gp/wishlist/    //Amazon屏蔽了商品愿望清单文件目录,但是对于其下的几个开放了权限,包括公开的愿望清单等

    Allow: /gp/wishlist/universal*

    Allow: /gp/wishlist/vendor-button*

    Allow: /gp/wishlist/ipad-install*

    Disallow: /registry/wishlist/


    Disallow: /gp/help/contact-us/general-questions.html*?type&email&skip=true

    Disallow: /gp/help/customer/accessibility?ie=UTF8&initialIssue=forgotpw&skip=true

    Disallow: /gp/registry/search.html

    Disallow: /gp/orc/rml/

    Disallow: /gp/digital/fiona/manage

    Disallow: /gp/entity-alert/external

    Disallow: /gp/customer-reviews/dynamic/sims-box

    Disallow: /review/dynamic/sims-box

    Disallow: /gp/redirect.html

    Disallow: /gp/customer-media/upload/

    Disallow: /gp/customer-media/actions/delete/

    Disallow: /gp/customer-media/actions/edit-caption/

    Disallow: /gp/dmusic/

    Disallow: /registry

    Disallow: /*/wishlist    //这里是禁止搜索引擎访问根目录下的所有wishlist目录文件

    Disallow: /gp/registry

    Disallow: /gp/aag

    Disallow: /gp/socialmedia/giveaways

    Disallow: /gp/aw/so.html    //禁止访问此目录下的so.html文件

    Disallow: /gp/pdp/profile/


    Disallow: /gp/help/customer/display.html*nodeId=200843370    //禁止访问节点编号为XXX的display.html文件,以下类似

    Disallow: /gp/help/customer/display.html*nodeId=200877580

    Disallow: /gp/help/customer/display.html*nodeId=200877590

    Disallow: /gp/help/customer/display.html*nodeId=200879080

    Disallow: /gp/help/customer/display.html*nodeId=200879100

    Disallow: /gp/help/customer/display.html*nodeId=200879120

    Disallow: /gp/help/customer/display.html*nodeId=200879160

    Disallow: /gp/help/customer/display.html*nodeId=200879140

    Disallow: /gp/help/customer/display.html*nodeId=200877610

    Disallow: /gp/help/customer/display.html*nodeId=200878960

    Disallow: /gp/help/customer/display.html*nodeId=200878980

    Disallow: /gp/help/customer/display.html*nodeId=200879000

    Disallow: /gp/help/customer/display.html*nodeId=200879040

    Disallow: /gp/help/customer/display.html*nodeId=200879020

    Disallow: /gp/help/customer/display.html*nodeId=200877630

    Disallow: /gp/help/customer/display.html*nodeId=200879200

    Disallow: /gp/help/customer/display.html*nodeId=200879220

    Disallow: /gp/help/customer/display.html*nodeId=200879240

    Disallow: /gp/help/customer/display.html*nodeId=200879280

    Disallow: /gp/help/customer/display.html*nodeId=200879260

    Disallow: /gp/help/customer/display.html*nodeId=200877650

    Disallow: /gp/help/customer/display.html*nodeId=200879320

    Disallow: /gp/help/customer/display.html*nodeId=200879340

    Disallow: /gp/help/customer/display.html*nodeId=200879360

    Disallow: /gp/help/customer/display.html*nodeId=200879400

    Disallow: /gp/help/customer/display.html*nodeId=200879380

    Disallow: /gp/help/customer/display.html*nodeId=200877560

    Disallow: /gp/help/customer/display.html*nodeId=200843460

    Disallow: /gp/help/customer/display.html*nodeId=200843440

    Disallow: /gp/help/customer/display.html*nodeId=200899270

    Disallow: /gp/help/customer/display.html*nodeId=200879440

    Disallow: /gp/help/customer/display.html*nodeId=200899330

    Disallow: /gp/help/customer/display.html*nodeId=200899350

    Disallow: /gp/help/customer/display.html*nodeId=200899390

    Disallow: /gp/help/customer/display.html*nodeId=200899410

    Disallow: /gp/help/customer/display.html*nodeId=200899430

    Disallow: /gp/help/customer/display.html*nodeId=200899220

    Disallow: /gp/help/customer/display.html*nodeId=200899450

    Disallow: /gp/help/customer/display.html*nodeId=200899670

    Disallow: /gp/help/customer/display.html*nodeId=200899530

    Disallow: /gp/help/customer/display.html*nodeId=200899470

    Disallow: /gp/help/customer/display.html*nodeId=200899550

    Disallow: /gp/help/customer/display.html*nodeId=200899570

    Disallow: /gp/help/customer/display.html*nodeId=200899510

    Disallow: /gp/help/customer/display.html*nodeId=200899610

    Disallow: /gp/help/customer/display.html*nodeId=200899630

    Disallow: /gp/help/customer/display.html*nodeId=200899650

    Disallow: /gp/help/customer/display.html*nodeId=200879180

    Disallow: /gp/help/customer/display.html*nodeId=200879060

    Disallow: /gp/help/customer/display.html*nodeId=200879300

    Disallow: /gp/help/customer/display.html*nodeId=200879420

    Disallow: /gp/help/customer/display.html*nodeId=200899290

    Disallow: /gp/help/customer/display.html*nodeId=200899310

    Disallow: /gp/help/customer/display.html*nodeId=200843380

    Disallow: /gp/help/customer/display.html*nodeId=200843420

    Disallow: /gp/help/customer/display.html*nodeId=200899230

    Disallow: /gp/help/customer/display.html*nodeId=200899250

    Disallow: /gp/help/customer/display.html*nodeId=200899370

    Disallow: /reviews/iframe

    Disallow: /gp/help/reports/infringement/jquery/handle-notice-submit.html

    Disallow: /gp/help/customer/handler/handle-email-submit.html

    Disallow: /ss/customer-reviews/lighthouse/

    Disallow: /gp/aw/ol/

    相关文章

      网友评论

          本文标题:EX05 Amazon 的Robots协议代码解释

          本文链接:https://www.haomeiwen.com/subject/jduzrftx.html