espi: Fix gpio_manage_callback() logic

espi_manage_callback() returns -EINVAL if it could not remove
callback. However if the list is empty success is returned when trying
to remove callback.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>

drivers/espi/espi_utils.h

@@ -32,6 +32,8 @@ static inline int espi_manage_callback(sys_slist_t *callbacks,
 				return -EINVAL;
 			}
 		}
+	} else if (!set) {
+		return -EINVAL;
 	}
 
 	if (set) {