Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zotero 7 and annotations not importing anymore #385

Open
bloosqr opened this issue Aug 16, 2024 · 27 comments
Open

Zotero 7 and annotations not importing anymore #385

bloosqr opened this issue Aug 16, 2024 · 27 comments

Comments

@bloosqr
Copy link

bloosqr commented Aug 16, 2024

With Zotero 6 and the integration plugin, annotations imported perfectly. With Zotero 7 annotations aren't showing up at all. If I use data explorer to look at all the entries annotations are empty. However if I open Zotero 6 and create annotations in Zotero 6, everything works perfectly.

What is odd about this is if I create an annotation in Zotero 7, Zotero 6 sees it but obsidian still will not import it in while connecting to Zotero 6 (e.g. seeing the annotation created in Zotero 7 but trying to import from Zotero 6).

The only way annotations are working currently is if I create new annotations in Zotero 6 and then import from Zotero 6, the obsidian plugin works.

I can't tell if this is a better bibtex bug or a Zotero integration bug but something is not talking to something properly with the new annotation approaches built into Zotero 7.

regards
avi

@stuarttravers
Copy link

I've just noticed this issue as well. No annotations being imported since upgrading to Zotero 7

@websurfer93
Copy link

websurfer93 commented Aug 21, 2024

I have the same issue. Annotations and even Zotero links are not importing anymore. (Version: 3.2.1)

@wdenton2
Copy link

Same issue here. No annotations since Zotero 7

@mgmeyers
Copy link
Owner

Hi all. Please uninstall and reinstall the better bibtex plugin and see if that fixes the issue for you.

@wdenton2
Copy link

wdenton2 commented Aug 21, 2024

Just tried, didn't seem to work. Opened a paper in the data explorer that has annotations and it still returns 0.

Edit: Tried uninstalling the obsidian plugin as well and doing a fresh install. Same issue occurs

@mgmeyers
Copy link
Owner

Hmm strange. Not really sure what could be going on. Annotations seem to be coming through just fine on my end.

@bloosqr
Copy link
Author

bloosqr commented Aug 21, 2024

No I have tried this as well. How are you pulling the annotations ? Is there something we can look at in a better bibtex export? The issue right now is data explorer is not showing any annotations that are generated in / with Zotero 7, so the template has nothing to import.

If it is helpful I am on a Mac OS X M2 machine.

@sikirebirth
Copy link

im not sure if im running into the same issue as everyone else but I'm not able to insert citations
image
please let me know if I should make a separate issue.

@NanamiKubota
Copy link

NanamiKubota commented Aug 22, 2024

Leaving a comment here as I am running into the same issues after updating from Zotero 6 to Zotero 7. I tried to uninstall and reinstall but the problem persists. The Zotero Data Explorer shows that I have 0 annotations even though the pdf is annotated. The other metadata appears to import just fine (see picture below).

Here are my version numbers for zotero and plugins (on macOS M1):

  • Zotero: 7.0.2
  • Zotero integration: 3.2.1
  • Better BibTex: 6.7.222

I also don't see an option to insert citations as mentioned by sikirebirth.

Screenshot 2024-08-22 at 3 08 28 PM

@mgmeyers
Copy link
Owner

@NanamiKubota Are there any annotations listed under either of the attachments?

Screenshot 2024-08-22 at 1 38 55 PM

@NanamiKubota
Copy link

Thanks for the quick reply! I do not see any annotations listed under the attachments:

Screenshot 2024-08-22 at 4 40 37 PM

@mgmeyers
Copy link
Owner

Hmm, the only other thing I can think of is to check if there are any errors in Zotero's error console after running an import thought the data explorer:

Screenshot 2024-08-22 at 2 24 34 PM

Also check if there are any errors in Obsidian's developer console:

Screenshot 2024-08-22 at 2 25 56 PM

@bloosqr
Copy link
Author

bloosqr commented Aug 22, 2024

As I was looking at this thread, I thought I'd try one more thing. I got myself the Zotero API and printed out the full dump of an item that I am trying to import using both Zotero 7 and Zotero 6 e.g. in a simple python script:

from pyzotero import zotero
#Initialize with the given library ID
library_id = 'XXXX'
library_type = 'user' # Assuming this is a personal library
api_key = "XXXXX"

#Create a Zotero instance
zot = zotero.Zotero(library_id, library_type, api_key)

#Retrieve the item by its ID

item_id ='UVI7FDW8'
item = zot.item(item_id)

#Retrieve the children of the item
children = zot.children(item_id)

#Print the item's data and its children
print("Item Data:")
print(item)
print("\nChildren:")
print(children)
~
~

What is bizarre is the output of that newly created entry in both Zotero 6 and 7 looked completely sensible to me (and the same) if I create the pdf and highlight the same section in both Zotero 6 and 7.

So then I tried reimporting the same entry from Zotero 7 with obsidian and lo and beyond it works now.

I have tried now to reproduce the lack of importing in Zotero 7 with new Zotero entries and old Zotero entries etc but now everything imports properly. Unfortunately I can't tell what fixed it (me running the script or just something else in my system that changed ) but on my side Zotero 7 and annotation importing seems to work now.

@NanamiKubota
Copy link

I can now get new highlights made in Zotero 7 (but not old annotation from Zotero 6) imported into Obsidian. Things that worked:

  1. First importing new documents via Zotero connector (Chrome extension) in Zotero 7, create new annotations, then import via Zotero Integration. This is good for new documents but problematic if you want to import old annotations from old documents made back in Zotero 6.

  2. If a document has multiple attachments, deleting attachments until there is only one PDF file remaining (see more detail below).

Things that do not work:

  1. Importing a document with multiple PDF attachments
  2. Importing a document with PDF + 'PubMed Central Link' attachment -> deleting the PubMed Central Link removes the import issue (not sure why this worked). The 'PubMed Central Link' attachment is a file that Zotero connector created back when I was using Zotero 6 and importing documents from PubMed.

Other issues:

  • Zotero Data Explorer now shows all of my highlights (both old highlights from when I annotated in Zotero 6 and new highlights in Zotero 7) using my custom template file after deleting the 'PubMed Central Link' attachment but when I actually go to import and create a new file using my template, it only shows the new highlights from Zotero 7.

@flogrows
Copy link

I too noticed yesterday that annotations weren't importing anymore. Similarly to @NanamiKubota, I've found it to be related to having multiple attachments to an item. All of my Zotero items have at least one extra attachment created by the Notero plugin.

My workaround is to temporarily move any extra attachments to the Zotero bin while I import my annotations to Obsidian, then restore them. This works fine - the annotations now appear in the Data Explorer and can be imported to a note.

I haven't noticed any issue with old/new annotations, but possibly because I don't tend to annotate within Zotero itself, but using another pdf editor, so the annotations are embedded in the pdf file.

Also:
I'm not sure that in my case the issue was caused by the update to Zotero 7, as I noticed the problem yesterday before I had updated. It was in looking around to see what might have changed that I noticed the pending update. There had also been an Obsidian update (via flathub) the day before.

@bloosqr
Copy link
Author

bloosqr commented Aug 23, 2024

I can confirm this now what others have said above as I just hit the same problem again and realized @flogrows and @NanamiKubota solution actually works for me. The issue is if there is another attachment in Zotero 7 nothing gets imported. (In fact I was getting hit by the pubmed link issue with the connector which had not been a problem with zot. 6) However if one removes the other attachment the importing works fine. If I delete or temporarily move the attachment during import I can import fine. (The reason I thought the problem was fixed was I was "testing" Zotero by just moving a single test PDF into zotero so did not have the attachments I typically get when I use Zotero connector to import pdfs from a web browser.

As an aside I wonder if we could also fix the multiple attachments issue with regards to annotations. I noticed with Zotero 6 the template would randomly pick one pdf (if there are supplementary pdfs to look at). It would be great if it scanned through all pdfs (or annotatable things) and imported all of those as part of the import.

@NanamiKubota
Copy link

Adding on to note that I finally was able to import both old annotations from Zotero 6 and new annotations from Zotero 7 using my custom template. Previously, I had issues where Zotero Data Explorer showed both new and old annotations for my custom template but when I went to import using my custom template, only the new annotations came up.

To get the import to work, I had to delete my old Obsidian entry from the Zotero 6 era and make a new entry via Zotero import, rather than appending new annotations to the old Obsidian document using the {% persist "annotations" %} line in my custom template.

@mgmeyers
Copy link
Owner

Ok, finally figured out what's going on here. Non-file attachments are causing errors in better bibtex (my fault). The next version of the Better Bibtex plugin will fix the issue.

@bloosqr
Copy link
Author

bloosqr commented Aug 24, 2024

Hey thank you so much for fixing this so proactively! Super appreciated!

@NanamiKubota
Copy link

Thank you so much! Really appreciate the time and effort. This plugin has been a life-saver!

@mgmeyers
Copy link
Owner

Ok, please update better bibtex and see if that fixes things for you.

@sikirebirth
Copy link

in regards to me mentioning citation not being available for anyone else wondering
image
In the options for the plugin I added a citation formatting style and is it now allowing citations
image
image

I apologize if this was mentioned elsewhere and I missed it.

@flogrows
Copy link

Ok, please update better bibtex and see if that fixes things for you.

Yes, all good now! Thanks so much :)

@stefanpricopie
Copy link

stefanpricopie commented Aug 28, 2024

I would assume the error below is related to the bibtex issues in this thread. I am getting the following error for any Zotero Integration action in Zotero 7. Zotero 6 on the other hand works fine for me

Screenshot 2024-08-28 at 12 57 20

Better BibTex 6.7.225
Zotero 7.0.2
Zotero Integration 3.2.1
Obsidian 1.6.7 (Installer 1.5.3)

Thank you for your help on this!

@AuLaSW
Copy link

AuLaSW commented Aug 28, 2024

Will support for annotations on non-PDF files be added? If I annotate, say, a webpage in Zotero 7 the annotations don't show up in Zotero Data Explorer, but the annotations for PDFs do.

@EureDudeheit
Copy link

I have Zotero 7.0.5 and betterbibtex 6.7.227 on Linux. When i want to import the notes in Obsidian, the error console in Zotero says:

`Error: ae:sql:get: undefined is not a string but a undefined 3 better-bibtex.js:17183:48

TypeError: this.gViewSourceUtils is undefined webconsole.js:223:5
viewSource resource://devtools/client/webconsole/webconsole.js:223
onViewSource resource://devtools/client/webconsole/service-container.js:43
onClick resource://devtools/client/shared/components/Frame.js:265
React 19
invokeGuardedCallbackImpl
invokeGuardedCallback
invokeGuardedCallbackAndCatchFirstError
executeDispatch
executeDispatchesInOrder
executeDispatchesAndRelease
executeDispatchesAndReleaseTopLevel
forEachAccumulated
runEventsInBatch
runExtractedEventsInBatch
handleTopLevel
batchedUpdates$1
batchedUpdates
dispatchEvent
1
unstable_runWithPriority
interactiveUpdates$1
interactiveUpdates
dispatchInteractiveEvent

`

@MyMarvel
Copy link

MyMarvel commented Nov 14, 2024

I have the same issue - cannot import my handwritten annotations into Obsidian
Obsidian 1.7.6, Zotero 7.0.9 (x64), Bibtext 6.7.251, Ubuntu 23.10 (and Windows 10 too! Just checked, the same issue)
When enabling bibtext logs and trying to import some zotero document into obsidian no errors in Zotero console :(

And by the way, here is a strange thing, I think it is related: I noticed a message "PDF utility is up to date" in the plugin settings, but I didn't install it beforehand! I think something is wrong in using this utility in Linux. Here is a screenshot:
image

And obviously I installed the utility and wrote a path to it in the settings, added all privileges to it etc, nothing was changed at all (the status still "PDF utility is up to date" and no images are imported)

When tested with Windows 10, found the following error log in Better Bibtext Debug Log:

[JavaScript Error: "uncaught exception: undefined" {file: "chrome://zotero/content/xpcom/file.js" line: 339}]
[JavaScript Error: "NotFoundError: No such JSWindowActor 'DevToolsFrame'"]
[JavaScript Error: "NotFoundError: No such JSWindowActor 'DevToolsFrame'"]

Please, help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests