There are countless design decisions involved in making a core, especially if it provides bootloader support.
This list is far from complete at this point in time.
I consider the following "rules" to be requirements for a usable core or bootloader (as appropriate).
Upload should never be able to softbrick or firmbrick a chip without native USB, while burn bootloader with improper settings may.
On native USB boards, a sufficiently bad piece of code can cause USB to fail to operate, necessitating other programming methods to restore functionality. This is largely unavoidable except through, well, not writing bad code that hoses USB (anything that triggers a dirty reset will do this).
This means that on classic AVRs, a core which writes to the fuses on upload is badly designed: All of the fuses (except, rarely, the extended ones) can break further ISP programming - the high fuse can result in a firm brick by disabling that programming interface, requiring HV programming, an onerous process. Plans and code for a "fuse doctor" exist for the tiny85 and 84 online, as these use HVSP. I have never heard of parts with more pins being restoredfrom a hard brick, which requires HVPP, a protocol requiring no less than 20 connections, generally forcing the part to be dismounted from the board to prevent interferance from connecteded devices. The low fuse can't result in such a severe brick, but if set to use a clock or crystal when one isn't connected, you are softbricked an a suitable clock (which can be generated by a modified arduino isp sketch) must be applied to the CLKI pin while performing ISP programming.
On modern AVRs, to enforce the same condition, a subset of the fuses can be set on all updi uploads, and probably should, another subset can brick the chip, and a third subset probably should only be reset to default, and only on burn bootloader; this is how my core works)
- Safe fuses that are a good idea to set are the CODESIZE/BOOTSIZE, or APPEND/BOOTEND fuses (so that you don't get wacky behavior when uploading to a board that used to have optiboot on it. Also, SYSCFG fuses that do NOT have a UPDI disabling function can be freely set. MVIO is not a hazard however - if MVIO is disabled, most of the MVIO behavior actually remains - just without any checks to make sure that the voltage is within range, and hence the MVIO pins behave unpredictably when VDDIO2 is not supplied, and MVIO state cannot be determined.
- Unsafe fuses include one of the SYSCFG fuses that control whether the UPDI pin is acting as UPDI (since they require an HV programmer to recover from), and the BODCFG fuse, which can set BOD to a voltage lower than the supply. If both the chip and everything else connected to it can take a voltage abovethe new BOD voltage, this is not a concern. But if they're not, this can't be recovered from without desoldering parts!
- Safe but unused fuses - these are fuses that are never set to anything but their default value by a core. I recommend setting them to that only when burning the bootloader, so that users can change them,
No code uploaded through a bootloder should ever be cabable of interfering with bootloader functionality
With the exception of native usb parts as noted above, there should be absolutely no way that uploaded code can trash the bootloader, short of a part (like classic tinyAVR) which does not have a hardware bootloader section (which can't be written from outside that section and uploaded code that straight up erases or scribbles over the bootloader (This is used to enable the vUSB bootloader update on Micronucleus boards).
This is - in my opinion, the most fundamental rule for bootloader design. Any bootloader that does not adhere to this rule is unfit for use in production. Examples include: Almost all classic AVR bootloaders, and many modern AVR ones, including unmodified versions of optiboot. The defect that is relevant here is that the reset flag register is used to determine whether the entry conditions of the bootloader are being met. if the bootloader does not clear the reset flag register, and the application does not either, the bootloader cannot accurately determine the case of the reset - or even whether a valid reset occurred as opposed to a dirty one. Since 99.9% of arduino code doesn't touch the reset flag registers.
__asm__ __volatile__("jmp 0x0000")
is evil. It generates a guaranteed dirty reset, and when using a bootloader, the bootloader is highly likely to malfunction. On classic AVRs it was commonly used as a (terribad) software reset. Modern AVRs have a software reset functiom. any modern AVR bootloader code that has not cleared the reset flags cannot detect this and is thus unfit for production use. Upon starting and seeing no reset flags, any modern AVR bootloader should always immediately issue a software reset, which will produce the behavior the idiot who wrote the application code wants, which is to start the bootloader from software. Whenever this is discovered in code intended to run on a modern AVR, it should be reported to the author as a bug.
It shalt not leave any register, for any reason in a non-default state unless clearly and explicitly documented, and these registers should not impact functioning (ie, it's acceptable to, having no other alternatives, pass the reset flags in a GPIOR. In the case of my cores, the first rule was addressed by storing the reset flag register values it saw at startup to GPIOR0, clearly documenting that this is done and reminding people hat they should thus clear it during setup(); megaTinyCore and DxCore both have hooks for founctions to replace the builtin handling of the reset flags. Remember that:
- POR clears all other reset flags.
- BOD clears all reset flags except POR. That means slow rising power will often leave this flag set if BOD is enabled. This is an example of why you need to make sure the reset flags are cleared. You may absolutely want to have POR be an entry condition. You almost certainly do not want a BOD reset that appears in the absence of PORF to be an entry condition, as it signifies a flaky power source, which is the last time you want to be writing to the flash!
- On classic AVRs, WDRF MUST be cleared immediately if detected, followed by a write to the watchdog control register to set it as desired. Classic AVRs reset by WDT will start up with WDT set to reset on shortest timeout, and must be immediately reconfigured. Modern AVRs don't do that.
- ALL OTHER RESETS ONLY ADD THEIR OWN RESET FLAG.
The best way, in my opinion to achieve this is to do two things:
- Use the watchdog timer reset to exit the bootloader, thus restoring ALL registers to their reset state.
- Make sure that WDRF is cleared and (on classic AVRs) WDT is turned off) before starting the app.
Some implementations would pass the reset flags in r2. This is bad. It requires that the application immediately check that register - except that normal sketch code only runs after the hardware is initialized; this may result in whichever working register you chose being overwritten in addition to requiring inline assembly to retrieve.