It is quite often during developing phase to install, try and then uninstall various contributed (or custom) modules. After the development is done and you want to clean things up and make sure no extra disabled modules are sitting around your drupal website, this is what you have to do.
drush pm-list --type=Module --status="disabled,not installed" --no-core
After running the above drush command, this is what i got. Now if you look carefully you will notice that there are some modules that come together bundled with their "parent" module. Like Toolbar and Admin Toolbar Links Access Filter (admin_toolbar_links_access_filter) etc. So before start typing composer remove drupal/whatever make sure that you are not trying to remove a "submodule" like this.
Package Name Status Version
--------------------------------- ----------------------------------------------------------------------- ---------- ---------------
Administration Admin Toolbar Links Access Filter (admin_toolbar_links_access_filter) Disabled 8.x-2.5
Administration Admin Toolbar Search (admin_toolbar_search) Disabled 8.x-2.5
Spam control CAPTCHA (captcha) Disabled 8.x-1.2
Spam control Image CAPTCHA (image_captcha) Disabled 8.x-1.2
Testing captcha long form id test module (captcha_long_form_id_test) Disabled 8.x-1.2
Testing Captcha Test (captcha_test) Disabled 8.x-1.2
Field types Color Field (color_field) Disabled 8.x-2.5
Chaos tool suite Chaos Tools (ctools) Disabled 8.x-3.7
Chaos tool suite (Experimental) Chaos Tools Blocks (ctools_block) Disabled 8.x-3.7
Other Entity Mask (ctools_entity_mask) Disabled 8.x-3.7
Chaos tool suite (Experimental) Chaos Tools Views (ctools_views) Disabled 8.x-3.7
Development Devel Generate (devel_generate) Disabled 4.1.1
Development Web Profiler (webprofiler) Disabled 4.1.1
Other Extra Field (extra_field) Disabled 8.x-1.3
Other Extra Field Example (extra_field_example) Disabled 8.x-1.3
Migration Field Group Migrate (field_group_migrate) Disabled 8.x-3.2
Fields Field Group (field_group) Disabled 8.x-3.2
Other Font Awesome Iconpicker Widget (fontawesome_iconpicker_widget) Disabled 8.x-2.19
Other Font Awesome Media Entity (fontawesome_media) Disabled 8.x-2.19
Field types Form Options Attributes (form_options_attributes) Disabled 8.x-1.2
