Ursel Z. used this system when sewing the "Golden October". In this case there is more than one fabric corresponding to each symbol.
Usually choosing fabrics takes a long time. Using a fabric chart will record the result clearly and you will not mix it up during block construction.
Mount the sheet close to your sewing space so you can see it easily and it will not be buried under the stuff on your table.