Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change FS from SPIFFS to LittleFS #684

Draft
wants to merge 7 commits into
base: develop
Choose a base branch
from

Conversation

gorec2005
Copy link

Just replace FS ... work for me...

@kleini
Copy link
Collaborator

kleini commented Jun 28, 2020

I use

pio run -t uploadfs

to upload my config.json for every device to SPIFFS. I don't use the config mode at all and I even use the compile flag to remove the code for the config mode. How do I upload the config.json to LittleFS?

@luebbe
Copy link
Collaborator

luebbe commented Jun 28, 2020

I also replaced all occurrences of Spiffs with LittleFS, but I see two pending problems:

  • Uploading to LittleFS is not yet supported by PlatformIO
  • Other libraries on which Homie depends, also use Spiffs and should be migrated too (alt least EspAsyncWebServer)

I'm pretty sure we shouldn't mix the two and I'm also pretty sure that we should put everything into defines at least for some time, so that people can have spiffs as default. After a while we could make LittleFS the default.

Which is why I haven't pursued the issue further (yet)

@kleini
Copy link
Collaborator

kleini commented Jun 28, 2020

closes #682

@gorec2005
Copy link
Author

gorec2005 commented Jul 7, 2020

Hi! i got work it with this... me-no-dev/ESPAsyncWebServer#792 (comment)
and now have trouble only with subdir's in ESP_AsyncFSBrowser all other work ok.

@MajorTwip
Copy link

Is LittleFS availlable on ESP32?

@dalbert2
Copy link

Is there a migration strategy for moving from SPIFFS to LittleFS? If SPIFFS is used in actual products, there is data stored in those products that can't just be thrown away if the software is upgraded.

@luebbe
Copy link
Collaborator

luebbe commented Nov 13, 2020

That is exactly the reason why I haven't merged anything yet. I remember coming across a project that worked on in-place migration of the data, but can't find it now.

@Maenthyx
Copy link

Hi,
I really enjoy homie-esp8266 since some days and would love to see this project to evolve. I wonder whether there are some news to this topic? May I add my2cents?
In my opinion, conversion of any existing filesystem/data on running systems is out-of-scope of this project. Adding code to support an automatic conversion from spiffs to littlefs (whatever this might look like) would result in (a lot of additional) code/dependencies which doesn't fit into the "Homie" scope. Have in mind that this would raise future efforts to maintain that "legacy code".
I also couldn't find any other project considering a conversion from spiffs to littlefs on running systems, so why should this project do?

Nevertheless, I feel the pain loosing data. So I'd suggest a process (and maybe tools/simple sketches) to download the data, update the system and upload the data. Either manually done, or by leveraging the OTA feature. Well documented, but not implemented into this project.

You also might want to read this thread, discussing this topic, options and efforts.
Start reading from "Feb 25 14:14": Gitter Thread

Also, for a limited time and depending on the effort, both spiffs and littlefs could be supported by using compiler flags, resulting in running either spiffs or littlefs (but not both at the same time). So people have some time to migrate their systems.

Thoughts?

@kleini
Copy link
Collaborator

kleini commented Dec 27, 2020

Supporting both filesystems through compiler flags was an earlier suggestion, too. This absolutely makes sense. I personally am still missing a tool to upload config.json to LittleFS filesystem, so I can test it.

@Maenthyx
Copy link

Maenthyx commented Dec 27, 2020

Afaik, ArduinoIDE supports uploading files to littlefs. But I'm not sure if it's already built in or a plugin is required.
I use PlatformIO, and by simply setting littlefs in the environment configuration you can upload files to littlefs using the pio cli.

May I ask what exactly you're missing? Don't you use ArduinoIDE or PlatformIO?

@kleini
Copy link
Collaborator

kleini commented Dec 27, 2020

I am using PlatformIO and did not have a look yet into PlatformIO and LittleFS. Good to hear, that it is supported.

@@ -85,11 +85,11 @@ bool Config::load() {
const char* reqWifiDns2 = reqWifi["dns2"] | "";

uint16_t reqMqttPort = reqMqtt["port"] | DEFAULT_MQTT_PORT;
bool reqMqttSsl = reqMqtt["ssl"] | false;
// bool reqMqttSsl = reqMqtt["ssl"] | false;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would break MQTT over SSL for me.
As this PR contains a mixture of multiple things:

  • ota_updater changes
  • LittleFS support
  • broken SSL support
  • changes on WiFi connect behaviour
    we should split this up into multiple smaller PRs covering each a single topic. I will take care about a compiler flags driven switch between SPIFFS and LittleFS.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, for the review. Maybe the ota_updater changes are caused by merging my changes back into his fork?
I also started working on the LittleFS change but stopped when it came to the async web server, since it doesn't support LittleFS. IMO LittleFS can right now only be supported with HOMIE_CONFIG=0. When the web server is used, we should default to SPIFFS.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not have a look at the commits and commit history but I think, we should try to keep Git history a little bit clean. For me code changes are already hard to follow.
Maybe I can make compile fail if LittleFS is selected without disabling the web server.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better with a clean start. If you give it a go using this PR as inspiration, we can close this one in a few days.
Failing the compilation should be easy with a set of nested #if(n)defs

@stritti stritti marked this pull request as draft December 28, 2020 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants