o J *j(. @sRdZddlZddlZddlZddlmZdZeZej ddddd d dRd e d e d e fddZ d e fddZdZdZdSde de d e fddZdZdTde de de d efddZd ZdUd"e d#e d$e d e fd%d&Zhd'Zegd(ed)d*Zd+e d e fd,d-ZdVd/ed0e d efd1d2Zd3e fd4d5Zd6e fd7d8Zed9kr%d:Z d;Z!dee e!d?Z$e$ree$ne#d@e#dAee ddBdCZ%e%ddDD]Z&e&dEpdddZ'e#dFe&dGdHe'dIqe#dJee dddCZ(ee(dKdLZ)e)D]Z*e#dFe*d"dMe*dNdOqe#dPe"D]Z+ee+e d!dQZ,ee,e-dqdSdS)Wuh 네이버 플레이스 크롤러 (GraphQL API 기반) - pcmap-api.place.naver.com/place/graphql 사용 N)datetimez/https://pcmap-api.place.naver.com/place/graphqlzoMozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36z*/*zko-KR,ko;q=0.9zapplication/jsonzhttps://pcmap.place.naver.com/)z User-AgentAcceptzAccept-Languagez Content-TypeRefererquery variablesreturnc Cs||pidg}z"tjt|dd}||}t|tr(|ddp&iWSiWStyE}zt d|iWYd}~Sd}~ww)N)rr )jsontimeoutrdataz [GQL ERROR] ) SESSIONpost GRAPHQL_URLraise_for_statusr isinstancelistget Exceptionprint)rrbodyrr er%/var/www/html/bicorn/place/crawler.py_gqls$rcCs(|durdStt|ddpdS)u'4,460' → 4460Nr,)intstrreplacestrip)srrr _parse_num#sr"z query getPlaceDetail($query: String!, $display: Int) { places(input: {query: $query, display: $display}) { items { id name category address roadAddress phone visitorReviewCount blogCafeReviewCount x y } } }zn query getPlaceRating($id: String!) { placeDetail(input: {id: $id}) { base { visitorReviewsScore } } }rplace_id hint_namec s>|p}tt|dd}|didg}tfdd|Dd}|sD|rDttdd}|didg}tfd d|Dd}|sPtd d iSttd i}|d ididdpfd} t|dd} t|dd} |dd|dd|dp|dd|ddt| | | t dd S)u place_id 에 해당하는 업체 기본정보 수집 hint_name: DB에 저장된 업체명 (검색 정확도 향상) rdisplayplacesitemsc3,|]}t|ddtkr|VqdSidrNrr.0itr#rr H*z$get_place_summary..Nc3r*r+r-r.r1rrr2Nr3z[WARN] place_id u" 검색 결과에서 찾지 못함r, placeDetailbasevisitorReviewsScoregvisitorReviewCountrblogCafeReviewCountnamercategory roadAddressaddressphone%Y-%m-%d %H:%M:%S) r#r:r;r=r>ratingvisitor_review_countblog_review_count collected_at) rPLACE_SUMMARY_QrnextrPLACE_RATING_Qr"floatrnowstrftime) r#r$ query_strr r)matcheddata2items2 rating_datar@v_countb_countrr1rget_place_summary>s2     rQa  query getReviews($id: String!, $page: Int, $size: Int) { visitorReviews(input: {businessId: $id, page: $page, size: $size, sort: "RECENT"}) { total items { id author { nickname } rating body visitCount created } } }2pagesizec Cstt|||d}|didg}g}|D]/}||dd|didd|d|d d|d d |d dtd dq|S)N)r,rTrUvisitorReviewsr)r,rauthornicknamer@r visitCountrcreatedr?) review_idrWr@content visit_count created_atrC)rREVIEW_QrappendrrHrI)r#rTrUr r)reviewsr0rrrget_place_reviews|s       rbz query getPlaceRank($query: String!, $display: Int) { places(input: {query: $query, display: $display}) { total items { id name category visitorReviewCount } } }keyword my_place_idmax_rankc Cstt||d}|di}|dg}|dd}d}g}t|dD]2\} } t| dd} || | | d d| d dt| d d| t|kd | t|krU| }q#||t||t d dS)Nr&r(r)totalrrRr,rr:r;r8)rankr#r:r; review_count is_my_placer?)rdmy_rank total_found competitors searched_at) rRANK_Qr enumeraterr`r"lenrrHrI) rdrerfr resultr)rgrhrmir0pidrrrget_place_ranks2        ru>6것곳더때또번수잘좀한갔어같은그냥너무다시다음되게매우아주없는예요완전왔어이었있고있는정말진짜하고하는했어 같아요 없어요 있어요가과나는도로를에와은을의이까지부터에서으로이고이나 이에요).ru 이고요u 이라고u 이라서u 이었어u 이에서u어요u아요u이에ru이며ru이라u이다rrru이랑u처럼u만큼u한테u에게ru 습니다u니다u어서u아서u네요u군요rrrrrrrrrrrrru랑u게u고u서TkeyreversewordcCsBtD]}||rt|t|dkr|dt| Sq|S)uR조사/어미 제거 → 어근 반환 (어근이 2자 미만이면 원형 유지)N) _KR_SUFFIXESendswithrq)rsuffixrrr _normalize_krs rr%ratop_ncCsi}|D]1}|ddp d}td|D] }|tvrqt|}|tvs)t|dkr*q||dd||<qqt|ddd d }d d |d|DS) Nr\ru [가-힣]{2,}rrrRcSs|dS)NrRr)xrrrsz)analyze_review_keywords..TrcSsg|] \}}||dqS))rdcountr)r/wcrrr sz+analyze_review_keywords..)rrefindall STOPWORDSrrqsortedr))rar word_countreviewr\rstem sorted_wordsrrranalyze_review_keywordssrdcCstdtd|dtd|dtd|dtd|d td |d d td |ddtd|dtddS)Nz3 ==================================================u 업체명 : r:u 카테고리 : r;u 주소 : r=u 전화 : r>u 별점(평균): r@u ⭐u 방문자리뷰: rA개u 수집시간 : rCz2==================================================)rr)rrrr print_summarys rrc Cstd|dd|d}td|rd|ndd|d d td |d gdd D] }|dr;dnd}td|dd|dd|dd |q3dS)Nu 검색어: [rd]rku 내 업체: u{}위u 순위권 밖u / 전체 rlru 상위 5개:rmr4rju ← 내 업체rz rhu위 r:u | 리뷰 ri)rrformat)rrhrmarkrrr print_ranks *.r__main__11830966u소들녘 광명점u광명 소고기u 광명 갈비u📍 기본정보 수집...)r$u 수집 실패u 📝 최신 리뷰 수집...r4)rTrUr\z rWz | z...u 🔑 키워드 분석... )rz: ru회u 📊 순위 확인...)rf)N)r)rRrS)rc)r%).__doc__rtimerequestsrrSessionr headersupdaterdictrrr"rDrFrQr_rrbrorurrrqrrrrr__name__ MY_PLACE_ID MY_PLACE_NAMEKEYWORDSrsummaryrarr all_reviewskwskkw rank_datasleeprrrrsr     / !