Python: Script Yon ekstrè Google Autosuggest nan tandans pou mo kle rechèch Tanporèman nich ou

Python Script Pou kaptire tandans Autosuggest

Tout moun renmen Google Tandans, men li yon ti jan difisil lè li rive Long Tail Keywords. Nou tout renmen ofisyèl la google tandans sèvis pou jwenn Sur sou konpòtman rechèch la. Sepandan, de bagay anpeche anpil moun sèvi ak li pou travay solid;

  1. Lè ou bezwen jwenn nouvo mo kle Tanporèman nich, gen se pa ase done sou Google Tandans 
  2. Mank de ofisyèl API pou fè demann Google tandans: Lè nou fè pou sèvi ak modil tankou pytrends, Lè sa a, nou gen yo sèvi ak serveurs prokurasyon, oswa nou jwenn bloke. 

Nan atik sa a, mwen pral pataje yon Python Script nou te ekri ekspòtasyon Des mo kle atravè Google Autosuggest.

Jwenn ak magazen rezilta Autosuggest sou tan 

Sipoze nou gen 1,000 mo kle Grenn pou voye bay Google Autosuggest. An retou, nou pral pwobableman jwenn alantou 200,000 longtail mo kle. Lè sa a, nou bezwen fè menm bagay la yon semèn pita epi konpare ansanbl sa yo pou reponn de kesyon:

  • Ki kesyon yo nouvo mo kle konpare ak dènye fwa a? Sa a se pwobableman ka a nou bezwen. Google panse kesyon sa yo ap vin pi enpòtan - lè nou fè sa, nou ka kreye pwòp solisyon Google Autosuggest nou an! 
  • Ki kesyon yo mo kle ankò tandans?

Script la se byen fasil, ak pi fò nan kòd la mwen pataje isit la. Kòd la mete ajou sove done ki sòti nan kouri sot pase yo ak konpare sijesyon yo sou tan. Nou evite baz done ki baze sou dosye tankou SQLite fè li senp - se konsa tout depo done a ap itilize CSV dosye anba a. Sa pèmèt ou enpòte dosye a nan Excel epi eksplore tandans mo kle nich pou biznis ou.

Itilize Script Piton sa a

  1. Antre nan seri mo kle pitit pitit ou a ki ta dwe voye nan otokonplete: keywords.csv
  2. Ajiste anviwònman yo Script pou bezwen ou:
    • LANG: default "en"
    • PEYI: default "nou"
  3. Orè script la kouri yon fwa chak semèn. Ou kapab tou kouri li manyèlman jan ou ta renmen.
  4. Sèvi ak keyword_suggestions.csv pou plis analiz:
    • first_seed: sa a se dat kote rechèch la parèt la pou premye fwa nan autosuggest la
    • last_seen: dat kote rechèch la te wè pou dènye fwa
    • se_nouvo: si first_seen == last_seen nou mete sa a Vrè - Jis filtre sou valè sa a yo ka resevwa rechèch yo nouvo tandans nan Google autosuggest la.

Isit la nan Kòd la Piton

# Pemavor.com Autocomplete Trends
# Author: Stefan Neefischer (stefan.neefischer@gmail.com)
import concurrent.futures
from datetime import date
from datetime import datetime
import pandas as pd
import itertools
import requests
import string
import json
import time

charList = " " + string.ascii_lowercase + string.digits

def makeGoogleRequest(query):
    # If you make requests too quickly, you may be blocked by google 
    time.sleep(WAIT_TIME)
    URL="http://suggestqueries.google.com/complete/search"
    PARAMS = {"client":"opera",
            "hl":LANGUAGE,
            "q":query,
            "gl":COUNTRY}
    response = requests.get(URL, params=PARAMS)
    if response.status_code == 200:
        try:
            suggestedSearches = json.loads(response.content.decode('utf-8'))[1]
        except:
            suggestedSearches = json.loads(response.content.decode('latin-1'))[1]
        return suggestedSearches
    else:
        return "ERR"

def getGoogleSuggests(keyword):
    # err_count1 = 0
    queryList = [keyword + " " + char for char in charList]
    suggestions = []
    for query in queryList:
        suggestion = makeGoogleRequest(query)
        if suggestion != 'ERR':
            suggestions.append(suggestion)

    # Remove empty suggestions
    suggestions = set(itertools.chain(*suggestions))
    if "" in suggestions:
        suggestions.remove("")
    return suggestions

def autocomplete(csv_fileName):
    dateTimeObj = datetime.now().date()
    #read your csv file that contain keywords that you want to send to google autocomplete
    df = pd.read_csv(csv_fileName)
    keywords = df.iloc[:,0].tolist()
    resultList = []

    with concurrent.futures.ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
        futuresGoogle = {executor.submit(getGoogleSuggests, keyword): keyword for keyword in keywords}

        for future in concurrent.futures.as_completed(futuresGoogle):
            key = futuresGoogle[future]
            for suggestion in future.result():
                resultList.append([key, suggestion])

    # Convert the results to a dataframe
    suggestion_new = pd.DataFrame(resultList, columns=['Keyword','Suggestion'])
    del resultList

    #if we have old results read them
    try:
        suggestion_df=pd.read_csv("keyword_suggestions.csv")
        
    except:
        suggestion_df=pd.DataFrame(columns=['first_seen','last_seen','Keyword','Suggestion'])
    
    suggestionCommon_list=[]
    suggestionNew_list=[]
    for keyword in suggestion_new["Keyword"].unique():
        new_df=suggestion_new[suggestion_new["Keyword"]==keyword]
        old_df=suggestion_df[suggestion_df["Keyword"]==keyword]
        newSuggestion=set(new_df["Suggestion"].to_list())
        oldSuggestion=set(old_df["Suggestion"].to_list())
        commonSuggestion=list(newSuggestion & oldSuggestion)
        new_Suggestion=list(newSuggestion - oldSuggestion)
         
        for suggest in commonSuggestion:
            suggestionCommon_list.append([dateTimeObj,keyword,suggest])
        for suggest in new_Suggestion:
            suggestionNew_list.append([dateTimeObj,dateTimeObj,keyword,suggest])
    
    #new keywords
    newSuggestion_df = pd.DataFrame(suggestionNew_list, columns=['first_seen','last_seen','Keyword','Suggestion'])
    #shared keywords with date update
    commonSuggestion_df = pd.DataFrame(suggestionCommon_list, columns=['last_seen','Keyword','Suggestion'])
    merge=pd.merge(suggestion_df, commonSuggestion_df, left_on=["Suggestion"], right_on=["Suggestion"], how='left')
    merge = merge.rename(columns={'last_seen_y': 'last_seen',"Keyword_x":"Keyword"})
    merge["last_seen"].fillna(merge["last_seen_x"], inplace=True)
    del merge["last_seen_x"]
    del merge["Keyword_y"]
    
    #merge old results with new results
    frames = [merge, newSuggestion_df]
    keywords_df =  pd.concat(frames, ignore_index=True, sort=False)
    # Save dataframe as a CSV file
    keywords_df['first_seen'] = pd.to_datetime(keywords_df['first_seen'])
    keywords_df = keywords_df.sort_values(by=['first_seen','Keyword'], ascending=[False,False])   
    keywords_df['first_seen']= pd.to_datetime(keywords_df['first_seen'])
    keywords_df['last_seen']= pd.to_datetime(keywords_df['last_seen'])
    keywords_df['is_new'] = (keywords_df['first_seen']== keywords_df['last_seen'])
    keywords_df=keywords_df[['first_seen','last_seen','Keyword','Suggestion','is_new']]
    keywords_df.to_csv('keyword_suggestions.csv', index=False)

# If you use more than 50 seed keywords you should slow down your requests - otherwise google is blocking the script
# If you have thousands of seed keywords use e.g. WAIT_TIME = 1 and MAX_WORKERS = 5
WAIT_TIME = 0.2
MAX_WORKERS = 20
# set the autocomplete language
LANGUAGE = "en"
# set the autocomplete country code - DE, US, TR, GR, etc..
COUNTRY="US"
# Keyword_seed csv file name. One column csv file.
#csv_fileName="keyword_seeds.csv"
CSV_FILE_NAME="keywords.csv"
autocomplete(CSV_FILE_NAME)
#The result will save in keyword_suggestions.csv csv file

Telechaje Script Piton an

Ki sa ou panse?

Sit sa a sèvi ak Akismet diminye Spam. Aprann kijan kòmantè ou a trete.