京东：User-agent: *
Disallow: /?* 禁止访问网站中的动态页面
Disallow: /pop/*.html 禁止访问/pop/目录下的所有以".html"为后缀的URL
Disallow: /pinpai/*.html?* 禁止访问/pinpai/目录下的所有以".html"为后缀的动态页面
User-agent: EtaoSpider
Disallow: / 禁止EtaoSpider访问网站
User-agent: HuihuiSpider
Disallow: / 禁止HuihuiSpider访问网站
User-agent: GwdangSpider
Disallow: /
User-agent: WochachaSpider
Disallow: / 禁止GwdangSpider、WochachaSpider访问网站
天猫：
User-agent: *
Disallow: / 禁止所有搜索引擎访问网站的任何部分
亚马逊：
User-agent: *
Disallow: /buycar
Disallow: /cart
Disallow: /checkout
Disallow: /class
Disallow: /com
Disallow: /common
Disallow: /css
Disallow: /dll
Disallow: /doc
Disallow: /dp/e-mail-friend/
Disallow: /dp/manual-submit/
Disallow: /dp/product-availability/
Disallow: /dp/rate-this-item/
Disallow: /dp/shipping/
Disallow: /dp/twister-update/
Disallow: /gp/aws/ssop
Disallow: /gp/cart
Disallow: /gp/css/homepage.html
Disallow: /gp/customer-reviews/common/du
Disallow: /gp/flex
Disallow: /gp/gfix
Disallow: /gp/history
Disallow: /gp/item-dispatch
Disallow: /gp/music/clipserve
Disallow: /gp/music/wma-pop-up
Disallow: /gp/offer-listing
Disallow: /gp/product/e-mail-friend
Disallow: /gp/product/product-availability
Disallow: /gp/product/rate-this-item
Disallow: /gp/recsradio
Disallow: /gp/slredirect
Disallow: /gp/twitter/
Disallow: /gp/vote
Disallow: /gp/voting/
Disallow: /gp/yourstore
Disallow: /inc
Disallow: /js
Disallow: /lib
Disallow: /mn/bookLookInsideApp
Disallow: /mn/checkInitApp
Disallow: /mn/checkoutAlertMsgApp
Disallow: /mn/checkoutredirectApp
Disallow: /mn/giftCardApp
Disallow: /mn/loginApplication
Disallow: /mn/loyaltyApp
Disallow: /mn/orderAddrApp
Disallow: /mn/orderCfmApp
Disallow: /mn/orderDetailApp
Disallow: /mn/orderFailApp
Disallow: /mn/orderHistoryApp
Disallow: /mn/orderModifyApp
Disallow: /mn/orderSummaryApp
Disallow: /mn/paymentRedriveApp
Disallow: /mn/recommendReviewApp
Disallow: /mn/releaseReviewApp
Disallow: /mn/reviewVoteApplication
Disallow: /mn/selectPaymentMethodApp
Disallow: /mn/selectShippingOpptionApplication
Disallow: /mn/shipmentTraceApp
Disallow: /mn/shoppingCartApplication
Disallow: /mn/tellFriend
Disallow: /mn/thankYouApplication
Disallow: /mn/virtualAccountApp
Disallow: /mn/yourAccountApp
Disallow: /paper
Disallow: /xml
Disallow: /youraccount
Disallow: /ap/signin 禁止访问以上目录下的内容
Disallow: /gp/registry/wishlist/
Disallow: /wishlist/
Allow: /wishlist/universal*
Allow: /wishlist/vendor-button*
Allow: /wishlist/get-button*
Disallow: /gp/wishlist/
Allow: /gp/wishlist/universal*
Allow: /gp/wishlist/vendor-button*
Allow: /gp/wishlist/ipad-install*
Disallow: /registry/wishlist/ 除了Allow里面*模糊匹配的网址之外该文件夹下其它内容都不可访问
Disallow: /gp/help/contact-us/general-questions.html*?type&email&skip=true
Disallow: /gp/help/customer/accessibility?ie=UTF8&initialIssue=forgotpw&skip=true
Disallow: /gp/registry/search.html
Disallow: /gp/orc/rml/
Disallow: /gp/digital/fiona/manage
Disallow: /gp/entity-alert/external
Disallow: /gp/customer-reviews/dynamic/sims-box
Disallow: /review/dynamic/sims-box
Disallow: /gp/redirect.html
Disallow: /gp/customer-media/upload/
Disallow: /gp/customer-media/actions/delete/
Disallow: /gp/customer-media/actions/edit-caption/
Disallow: /gp/dmusic/
Disallow: /registry
Disallow: /*/wishlist
Disallow: /gp/registry
Disallow: /gp/aag
Disallow: /gp/socialmedia/giveaways
Disallow: /gp/aw/so.html
Disallow: /gp/pdp/profile/
Disallow: /gp/help/customer/display.html*nodeId=200843370
Disallow: /gp/help/customer/display.html*nodeId=200877580
Disallow: /gp/help/customer/display.html*nodeId=200877590
Disallow: /gp/help/customer/display.html*nodeId=200879080
Disallow: /gp/help/customer/display.html*nodeId=200879100
Disallow: /gp/help/customer/display.html*nodeId=200879120
Disallow: /gp/help/customer/display.html*nodeId=200879160
Disallow: /gp/help/customer/display.html*nodeId=200879140
Disallow: /gp/help/customer/display.html*nodeId=200877610
Disallow: /gp/help/customer/display.html*nodeId=200878960
Disallow: /gp/help/customer/display.html*nodeId=200878980
Disallow: /gp/help/customer/display.html*nodeId=200879000
Disallow: /gp/help/customer/display.html*nodeId=200879040
Disallow: /gp/help/customer/display.html*nodeId=200879020
Disallow: /gp/help/customer/display.html*nodeId=200877630
Disallow: /gp/help/customer/display.html*nodeId=200879200
Disallow: /gp/help/customer/display.html*nodeId=200879220
Disallow: /gp/help/customer/display.html*nodeId=200879240
Disallow: /gp/help/customer/display.html*nodeId=200879280
Disallow: /gp/help/customer/display.html*nodeId=200879260
Disallow: /gp/help/customer/display.html*nodeId=200877650
Disallow: /gp/help/customer/display.html*nodeId=200879320
Disallow: /gp/help/customer/display.html*nodeId=200879340
Disallow: /gp/help/customer/display.html*nodeId=200879360
Disallow: /gp/help/customer/display.html*nodeId=200879400
Disallow: /gp/help/customer/display.html*nodeId=200879380
Disallow: /gp/help/customer/display.html*nodeId=200877560
Disallow: /gp/help/customer/display.html*nodeId=200843460
Disallow: /gp/help/customer/display.html*nodeId=200843440
Disallow: /gp/help/customer/display.html*nodeId=200899270
Disallow: /gp/help/customer/display.html*nodeId=200879440
Disallow: /gp/help/customer/display.html*nodeId=200899330
Disallow: /gp/help/customer/display.html*nodeId=200899350
Disallow: /gp/help/customer/display.html*nodeId=200899390
Disallow: /gp/help/customer/display.html*nodeId=200899410
Disallow: /gp/help/customer/display.html*nodeId=200899430
Disallow: /gp/help/customer/display.html*nodeId=200899220
Disallow: /gp/help/customer/display.html*nodeId=200899450
Disallow: /gp/help/customer/display.html*nodeId=200899670
Disallow: /gp/help/customer/display.html*nodeId=200899530
Disallow: /gp/help/customer/display.html*nodeId=200899470
Disallow: /gp/help/customer/display.html*nodeId=200899550
Disallow: /gp/help/customer/display.html*nodeId=200899570
Disallow: /gp/help/customer/display.html*nodeId=200899510
Disallow: /gp/help/customer/display.html*nodeId=200899610
Disallow: /gp/help/customer/display.html*nodeId=200899630
Disallow: /gp/help/customer/display.html*nodeId=200899650
Disallow: /gp/help/customer/display.html*nodeId=200879180
Disallow: /gp/help/customer/display.html*nodeId=200879060
Disallow: /gp/help/customer/display.html*nodeId=200879300
Disallow: /gp/help/customer/display.html*nodeId=200879420
Disallow: /gp/help/customer/display.html*nodeId=200899290
Disallow: /gp/help/customer/display.html*nodeId=200899310
Disallow: /gp/help/customer/display.html*nodeId=200843380
Disallow: /gp/help/customer/display.html*nodeId=200843420
Disallow: /gp/help/customer/display.html*nodeId=200899230
Disallow: /gp/help/customer/display.html*nodeId=200899250
Disallow: /gp/help/customer/display.html*nodeId=200899370
Disallow: /reviews/iframe
Disallow: /gp/help/reports/infringement/jquery/handle-notice-submit.html
Disallow: /gp/help/customer/handler/handle-email-submit.html 禁止访问以上的html...
亚马逊具体的有点看不下去，也没有找到相关的解释，只能稍微看一下学习学习了...
网友评论