Todoman

The iCalendar format also supports saving tasks in form of VTODO-entries, with the same file extension as normal events: .ics. Many CalDAV servers support synchronizing tasks, vdirsyncer does too.

todoman is a CLI task manager supporting vdir. Its interface is similar to the ones of Taskwarrior or the todo.txt CLI app. You can use filesystem with it.