获取网页信息

jmao1234 · 发表于 2024-4-23 10:01:55

想通过python或者易语言，爬取网页里面的公司名称和公司信息，请高手指点！谢谢
url = https://www.chinaplasonline.com/ ... Exh&page=2&fromAd=0

zz1314 · 发表于 2024-4-23 10:10:37

url???

jmao1234 · 发表于 2024-4-23 10:20:37

zz1314 发表于 2024-4-23 10:10
url???

https://www.chinaplasonline.com/eMarketplace/AllExhibitors/simp?Tags=1063&Regions=44&src=31&SearchType=all&SearchBy=Exh&page=6&fromAd=0

szweb01 · 发表于 2024-4-23 10:28:11

网页_取网页文本（）精易模块的命令

cherryy · 发表于 2024-4-23 10:28:37

https://e.adsale.com.hk/WebCoreAPI_Search/OpenSearch/Search
找到这个包就可以提出来，全部信息都有

jmao1234 · 发表于 2024-4-23 10:28:57

szweb01 发表于 2024-4-23 10:28
网页_取网页文本（）精易模块的命令

好像读取不出来

jmao1234 · 发表于 2024-4-23 10:33:50

cherryy 发表于 2024-4-23 10:28
https://e.adsale.com.hk/WebCoreAPI_Search/OpenSearch/Search
找到这个包就可以提出来，全部信息都有
...

额，问题是不知道如何找这个包，请教！！

cherryy · 发表于 2024-4-23 10:36:03

jmao1234 发表于 2024-4-23 10:33
额，问题是不知道如何找这个包，请教！！

浏览器F12抓包

zz1314 · 发表于 2024-4-23 10:57:11

import requests, json

url = "https://e.adsale.com.hk/WebCoreAPI_Search/OpenSearch/Search"
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36',
"Content-Type": "application/json"
}

data = json.dumps({
"log": {
      "LangId": "936",
      "SearchBy": "Exh",
      "Keyword": "龙宇",
      "IP": "113.132.179.196",
      "Page": 0,
      "VistorGuid": "",
      "AdsaleGuid": "3a0ca70ff033454d9ba6d28a44b35adf",
      "SourceName": "eMarketpalce",
      "SelectedCriterion": {
         "Prods": [
            ""
         ],
         "Apps": [
            ""
         ],
         "Tags": [
            "1063"
         ],
         "Regions": [
            "44"
         ],
         "TechAreas": [
            ""
         ],
         "ThemeZone": [
            ""
         ],
         "Halls": [
            ""
         ]
      }
},
"model": {
      "summary": "summary_field:exhibitnamesc,summary_len:3000;summary_field:companynamesc,summary_len:3000;summary_field:exhibitdescsc,summary_len:3000;summary_field:producttagsc,summary_len:3000;summary_field:companycatsc,summary_len:3000;summary_field:companyappsc,summary_len:3000;summary_field:companydescsc,summary_len:3000;summary_field:companytagsc,summary_len:3000;",
      "Start": "0",
      "Size": "15",
      "Fetch_fields": "key;diycompid;companynamesc;sponsorid;sgrade;boothno;companytagid;companycatid;companycatsc;companyappid;companyappsc;liveurlsc;liveTitlesc;videofilename;logofilename;companydescsc;companytagsc;r_companytag;r_producttag;r_companycat;r_companyapp",
      "Query": "(comp_info_a:'龙宇' OR comp_info_b:'龙宇' OR comp_catapptagkey:'龙宇') AND seqid:(-1,)&&distinct=dist_key:diycompid,dist_count:1,dist_times:1,reserved:false,update_total_hit:true&&kvpairs=duniqfield:diycompid&&aggregate=group_key:companycatid,agg_fun:count(),max_group:4;group_key:exhibitid,agg_fun:count(),max_group:99999&&filter=(companycountryeformid=\"44\") AND (companytagid=1063 OR producttagid=1063)",
      "second_rank_name": "comp_adv",
      "first_rank_name": "sys_first_default",
      "Disable": "qp:spell_check",
      "raw_query": ""
}
})

res = requests.post(url=url, headers=headers, data=data)

print(res.json())
# 这个是cha询接口请根据实际需求修改个别参数获取结果即可

冰墩墩 · 发表于 2024-4-23 11:06:05

直接发送POST包就可以了刚试过了可以的

		自动登录	找回密码
密码			注册

[python] 获取网页信息