-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added Google Scholar Integration #7278
Open
xnnxnyx
wants to merge
13
commits into
langchain-ai:main
Choose a base branch
from
xnnxnyx:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+205
−0
Open
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
7526b55
chore: added file for google_scholar
xnnxnyx 02b76b1
chore: edited to use SerpAPI
xnnxnyx 58aa415
chore: added google_scholar example
xnnxnyx e3172bb
chore: updated google_scholar import in examples
xnnxnyx 882b735
chore: added google_scholar files
xnnxnyx 5caa2e5
chore: added test file for google_scholar
xnnxnyx f0dbfc1
chore: updated the format of the results in google_scholar
xnnxnyx e753ce5
chore: added google_scholar api
xnnxnyx 9a05b16
Merge branch 'langchain-ai:main' into google-scholar-final
xnnxnyx 94921f8
Merge pull request #1 from xnnxnyx/google-scholar-final
xnnxnyx 45d6f58
chore: updated google_scholar doc
xnnxnyx 9537edc
Merge pull request #2 from xnnxnyx/google-scholar-integration
xnnxnyx 4beb710
Merge branch 'langchain-ai:main' into main
xnnxnyx File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
--- | ||
hide_table_of_contents: true | ||
--- | ||
|
||
import CodeBlock from "@theme/CodeBlock"; | ||
|
||
# Google Scholar Tool | ||
|
||
The Google Scholar Tool enables your agent to access scholarly articles and publications data through the Google Scholar API. Use it to retrieve article titles, abstracts, authors, and publication information directly from Google Scholar. | ||
|
||
## Setup | ||
|
||
You will need to get an API key from [SerpAPI](https://serpapi.com/), which offers a Google Scholar API as part of its services. Sign up on SerpAPI, retrieve your API key from your account settings, and set it as process.env.SERPAPI_API_KEY or pass it in as an apiKey constructor argument. | ||
|
||
## Usage | ||
|
||
import IntegrationInstallTooltip from "@mdx_components/integration_install_tooltip.mdx"; | ||
|
||
<IntegrationInstallTooltip></IntegrationInstallTooltip> | ||
|
||
```bash npm2yarn | ||
npm install @langchain/openai @langchain/community @langchain/core | ||
``` | ||
|
||
import ToolExample from "@examples/tools/google_scholar.ts"; | ||
|
||
<CodeBlock language="typescript">{ToolExample}</CodeBlock> | ||
|
||
## Related | ||
|
||
- Tool [conceptual guide](/docs/concepts/tools) | ||
- Tool [how-to guides](/docs/how_to/#tools) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
import { GoogleScholarAPI } from "@langchain/community/tools/google_scholar"; | ||
|
||
const scholar = new GoogleScholarAPI({ | ||
apiKey: process.env.SERPAPI_API_KEY | ||
}); | ||
|
||
(async () => { | ||
try { | ||
const query = "Attention is all we need"; | ||
const results = await scholar._call(query); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Use |
||
console.log("Search Results:", results); | ||
} catch (error) { | ||
console.error("Error querying Google Scholar via SerpApi:", error); | ||
} | ||
})(); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,122 @@ | ||
import { Tool } from "@langchain/core/tools"; | ||
import { getEnvironmentVariable } from "@langchain/core/utils/env"; | ||
import fetch from "node-fetch"; // For making HTTP requests | ||
|
||
/** | ||
* Interface for parameters required by the GoogleScholarAPI class. | ||
*/ | ||
export interface GoogleScholarAPIParams { | ||
/** | ||
* Optional API key for accessing the SerpApi service. | ||
*/ | ||
apiKey?: string; | ||
} | ||
|
||
/** | ||
* Tool for querying Google Scholar using the SerpApi service. | ||
*/ | ||
export class GoogleScholarAPI extends Tool { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's call this |
||
/** | ||
* Specifies the name of the tool, used internally by LangChain. | ||
*/ | ||
static lc_name() { | ||
return "GoogleScholarAPI"; | ||
} | ||
|
||
/** | ||
* Returns a mapping of secret environment variable names to their usage in the tool. | ||
* @returns {object} Mapping of secret names to their environment variable counterparts. | ||
*/ | ||
get lc_secrets(): { [key: string]: string } | undefined { | ||
return { | ||
apiKey: "SERPAPI_API_KEY", | ||
}; | ||
} | ||
|
||
// Name of the tool, used for logging or identification within LangChain. | ||
name = "google_scholar"; | ||
|
||
// The API key used for making requests to SerpApi. | ||
protected apiKey: string; | ||
|
||
/** | ||
* Description of the tool for usage documentation. | ||
*/ | ||
description = `A wrapper around Google Scholar API via SerpApi. Useful for querying academic | ||
articles and papers by keywords or authors. Input should be a search query string.`; | ||
|
||
/** | ||
* Constructs a new instance of GoogleScholarAPI. | ||
* @param fields - Optional parameters including an API key. | ||
*/ | ||
constructor(fields?: GoogleScholarAPIParams) { | ||
super(...arguments); | ||
|
||
// Retrieve API key from fields or environment variables. | ||
const apiKey = | ||
fields?.apiKey ?? getEnvironmentVariable("SERPAPI_API_KEY"); | ||
|
||
// Throw an error if no API key is found. | ||
if (!apiKey) { | ||
throw new Error( | ||
`SerpApi key not set. You can set it as "SERPAPI_API_KEY" in your environment variables.` | ||
); | ||
} | ||
this.apiKey = apiKey; | ||
} | ||
|
||
/** | ||
* Makes a request to SerpApi for Google Scholar results. | ||
* @param input - Search query string. | ||
* @returns A JSON string containing the search results. | ||
* @throws Error if the API request fails or returns an error. | ||
*/ | ||
async _call(input: string): Promise<string> { | ||
// Construct the URL for the API request. | ||
const url = `https://serpapi.com/search.json?q=${encodeURIComponent( | ||
input | ||
)}&engine=google_scholar&api_key=${this.apiKey}`; | ||
|
||
// Make an HTTP GET request to the SerpApi service. | ||
const response = await fetch(url); | ||
|
||
// Handle non-OK responses by extracting the error message. | ||
if (!response.ok) { | ||
let message; | ||
try { | ||
const json = await response.json(); // Attempt to parse the error response. | ||
message = json.error; // Extract the error message from the response. | ||
} catch (error) { | ||
// Handle cases where the response isn't valid JSON. | ||
message = | ||
"Unable to parse error message: SerpApi did not return a JSON response."; | ||
} | ||
// Throw an error with detailed information about the failure. | ||
throw new Error( | ||
`Got ${response.status}: ${response.statusText} error from SerpApi: ${message}` | ||
); | ||
} | ||
|
||
// Parse the JSON response from SerpApi. | ||
const json = await response.json(); | ||
|
||
// Transform the raw response into a structured format. | ||
const results = | ||
json.organic_results?.map((item: any) => ({ | ||
title: item.title, // Title of the article or paper. | ||
link: item.link, // Direct link to the article or paper. | ||
snippet: item.snippet, // Brief snippet or description. | ||
publication_info: item.publication_info?.summary | ||
?.split(" - ") // Split the summary at hyphens. | ||
.slice(1) // Remove the authors from the start of the string. | ||
.join(" - ") ?? "", // Rejoin remaining parts as publication info. | ||
authors: item.publication_info?.authors | ||
?.map((author: any) => author.name) // Extract the list of author names. | ||
.join(", ") ?? "", // Join author names with a comma. | ||
total_citations: item.inline_links?.cited_by?.total ?? "", // Total number of citations. | ||
})) ?? `No results found for ${input} on Google Scholar.`; | ||
|
||
// Return the results as a formatted JSON string. | ||
return JSON.stringify(results, null, 2); | ||
} | ||
} |
36 changes: 36 additions & 0 deletions
36
libs/langchain-community/src/tools/tests/google_scholar.int.test.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
import { test, expect, describe } from "@jest/globals"; | ||
import { GoogleScholarAPI } from "../google_scholar.js"; | ||
|
||
describe("GoogleScholarAPI", () => { | ||
test("should be setup with correct parameters", async () => { | ||
const instance = new GoogleScholarAPI({ | ||
apiKey: process.env.SERPAPI_API_KEY | ||
}); | ||
expect(instance.name).toBe("google_scholar"); | ||
}); | ||
|
||
test("GoogleScholarAPI returns a string for valid query", async () => { | ||
const tool = new GoogleScholarAPI({ | ||
apiKey: process.env.SERPAPI_API_KEY | ||
}); | ||
const result = await tool.invoke("Artificial Intelligence"); | ||
expect(typeof result).toBe("string"); | ||
}); | ||
|
||
test("GoogleScholarAPI returns non-empty string for valid query", async () => { | ||
const tool = new GoogleScholarAPI({ | ||
apiKey: process.env.SERPAPI_API_KEY | ||
}); | ||
const result = await tool.invoke("Artificial Intelligence"); | ||
expect(result.length).toBeGreaterThan(0); | ||
}); | ||
|
||
test("GoogleScholarAPI returns 'No results found' for bad query", async () => { | ||
const tool = new GoogleScholarAPI({ | ||
apiKey: process.env.SERPAPI_API_KEY | ||
}); | ||
const result = await tool.invoke("dsalkfjsdlfjasdflasdl"); | ||
expect(result).toBe("\"No results found for dsalkfjsdlfjasdflasdl on Google Scholar.\""); | ||
}); | ||
|
||
}); |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please use the template here?
https://github.com/langchain-ai/langchainjs/tree/main/libs/langchain-scripts/src/cli/docs/templates