diff --git a/source/operations/concepts/thresholds.rst b/source/operations/concepts/thresholds.rst index cc75d644..e8c96cb0 100644 --- a/source/operations/concepts/thresholds.rst +++ b/source/operations/concepts/thresholds.rst @@ -59,7 +59,7 @@ S3 API Limits * - Maximum length for each ``/`` separated object name segment - 255 - * - Maximum number of object verions for a unique object + * - Maximum number of object versions for a unique object - 10000 (Configurable) Erasure Code Limits diff --git a/source/reference/minio-server/settings/core.rst b/source/reference/minio-server/settings/core.rst index 2c68c8ee..31fd99a6 100644 --- a/source/reference/minio-server/settings/core.rst +++ b/source/reference/minio-server/settings/core.rst @@ -20,8 +20,8 @@ This page covers settings that control core behavior of the MinIO process. :start-after: start-minio-settings-test-before-prod :end-before: end-minio-settings-test-before-prod -MinIO Options -~~~~~~~~~~~~~ +MinIO Server CLI Options +------------------------ .. tab-set:: @@ -47,12 +47,9 @@ For example, to set up ftp access, you could set the variable to something like On Unix-like systems, you can save a file with the environment variable to ``/etc/defaults/minio`` instead of setting the variable manually. -Common Settings +Storage Volumes --------------- -Volumes -~~~~~~~ - .. tab-set:: .. tab-item:: Environment Variable @@ -71,7 +68,7 @@ Volumes :end-before: end-minio-settings-no-config-option Environment Variable File Path -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------------ .. tab-set:: @@ -90,7 +87,7 @@ Environment Variable File Path :end-before: end-minio-settings-no-config-option Workers for Expiration -~~~~~~~~~~~~~~~~~~~~~~ +---------------------- .. tab-set:: @@ -110,7 +107,7 @@ Workers for Expiration :end-before: end-minio-settings-no-config-option Domain -~~~~~~ +------ .. tab-set:: @@ -145,7 +142,7 @@ Domain .. _minio-scanner-speed-options: Scanner Speed -~~~~~~~~~~~~~ +------------- .. tab-set:: @@ -196,74 +193,55 @@ All of the settings in this section fall under the following top-level key: .. mc-conf:: compression -Allow Encryption -~~~~~~~~~~~~~~~~ +Enable Compression +~~~~~~~~~~~~~~~~~~ .. tab-set:: .. tab-item:: Environment Variable :sync: envvar - .. envvar:: MINIO_COMPRESSION_ALLOW_ENCRYPTION + .. envvar:: MINIO_COMPRESSION_ENABLE .. tab-item:: Configuration Setting :sync: config - .. mc-conf:: compression allow_encryption + .. mc-conf:: compression enable :delimiter: " " *Optional* -Set to ``on`` to encrypt objects after compressing them. +Set to ``on`` to enable data compression for new objects. Defaults to ``off``. -.. admonition:: Encrypting compressed objects may compromise security - :class: warning - - MinIO strongly recommends against encrypting compressed objects. - If you require encryption, carefully evaluate the risk of potentially leaking information about the contents of encrypted objects. +Enabling or disabling data compression does not change existing objects. -Enable Compression -~~~~~~~~~~~~~~~~~~ +Allow Encryption +~~~~~~~~~~~~~~~~ .. tab-set:: .. tab-item:: Environment Variable :sync: envvar - .. envvar:: MINIO_COMPRESSION_ENABLE + .. envvar:: MINIO_COMPRESSION_ALLOW_ENCRYPTION .. tab-item:: Configuration Setting :sync: config - .. mc-conf:: compression enable + .. mc-conf:: compression allow_encryption :delimiter: " " *Optional* -Set to ``on`` to enable data compression for new objects. +Set to ``on`` to encrypt objects after compressing them. Defaults to ``off``. -Enabling or disabling data compression does not change existing objects. - -Comments -~~~~~~~~ - -.. tab-set:: - - .. tab-item:: Environment Variable - - This setting does not have an environment variable option. - Use the configuration variable instead. - - .. tab-item:: Configuration Setting - :selected: - - .. envvar:: compression comment - -*Optional* +.. admonition:: Encrypting compressed objects may compromise security + :class: warning -Specify a comment to associate with the data compression configuration. + MinIO strongly recommends against encrypting compressed objects. + If you require encryption, carefully evaluate the risk of potentially leaking information about the contents of encrypted objects. Compression Extensions ~~~~~~~~~~~~~~~~~~~~~~ @@ -287,12 +265,11 @@ Comma-separated list of the file extensions to compress. Setting a new list of file extensions replaces the previously configured list. Defaults to ``".txt, .log, .csv, .json, .tar, .xml, .bin"``. -.. admonition:: Default excluded files - :class: note +.. versionchanged:: RELEASE.2024-03-15T01-07-19Z - Some types of files cannot be significantly reduced in size. - MinIO will *not* compress these, even if specified in an :mc-conf:`~compression.extensions` argument. - See :ref:`Excluded types ` for details. + Specify ``"*"`` to direct MinIO to compress all supported file types. + +MinIO does not support compressing file types on the :ref:`Excluded File Types ` list, even if explicitly specified in this argument. Compression MIME Types ~~~~~~~~~~~~~~~~~~~~~~ @@ -323,8 +300,27 @@ Defaults to ``"text/*, application/json, application/xml, binary/octet-stream"`` MinIO will *not* compress these, even if specified in an :mc-conf:`~compression.mime_types` argument. See :ref:`Excluded types ` for details. +Comments +~~~~~~~~ + +.. tab-set:: + + .. tab-item:: Environment Variable + + This setting does not have an environment variable option. + Use the configuration setting instead. + + .. tab-item:: Configuration Setting + :selected: + + .. envvar:: compression comment + +*Optional* + +Specify a comment to associate with the data compression configuration. + Erasure Stripe Size -~~~~~~~~~~~~~~~~~~~ +------------------- .. tab-set:: @@ -357,5 +353,30 @@ The selected stripe size is **immutable** after the cluster has been initialized MinIO's stripe selection algorithms set appropriate defaults for the majority of workloads. Changing the stripe size from this default is unusual and generally not necessary or advised. +Maximum Object Versions +----------------------- + +.. tab-set:: + + .. tab-item:: Environment Variable + :sync: envvar + + .. envvar:: MINIO_API_OBJECT_MAX_VERSIONS + + .. tab-item:: Configuration Setting + :sync: config + + .. mc-conf:: api object_max_versions + :delimiter: " " + +*Optional* + +Overrides the default maximum version per object limit of ``10000`` with the user specified value. + +.. important:: + + The default limit of 10,000 provides a safety valve against incorrect or inefficient application behavior in versioned buckets. + Lifting this limit without first ensuring your applications are designed for versioned operations may result in a negative performance impact over time. + diff --git a/source/reference/minio-server/settings/storage-class.rst b/source/reference/minio-server/settings/storage-class.rst index 536b50a1..c0201b34 100644 --- a/source/reference/minio-server/settings/storage-class.rst +++ b/source/reference/minio-server/settings/storage-class.rst @@ -14,6 +14,17 @@ Erasure Code Settings This page covers settings that configure the :ref:`Erasure Code ` :ref:`parity ` to use for objects written to the MinIO cluster. This impacts how MinIO uses the space on the drive(s) and how MinIO can recover objects stored on lost drives or similar issues. +.. include:: /includes/common-mc-admin-config.rst + :start-after: start-minio-settings-defined + :end-before: end-minio-settings-defined + +.. include:: /includes/common-mc-admin-config.rst + :start-after: start-minio-settings-test-before-prod + :end-before: end-minio-settings-test-before-prod + +Standard Storage Class +---------------------- + .. note:: *MinIO Storage Classes* are distinct from *AWS Storage Classes*. @@ -23,80 +34,140 @@ This impacts how MinIO uses the space on the drive(s) and how MinIO can recover For tiering from one type of storage to another, such as for cost management purposes, see :ref:`minio-lifecycle-management-tiering`. -Define any of these environment variables in the host system prior to starting or restarting the MinIO process. -Refer to your operating system's documentation for how to define an environment variable. +.. tab-set:: -.. include:: /includes/common-mc-admin-config.rst - :start-after: start-minio-settings-test-before-prod - :end-before: end-minio-settings-test-before-prod + .. tab-item:: Environment Variable + :sync: envvar + + .. envvar:: MINIO_STORAGE_CLASS_STANDARD + + .. tab-item:: Configuration Setting + :sync: config + + .. include:: /includes/common-mc-admin-config.rst + :start-after: start-minio-settings-no-config-option + :end-before: end-minio-settings-no-config-option + +The :ref:`parity level ` for the deployment. +MinIO shards objects written with the default ``STANDARD`` storage class using this parity value. + +MinIO references the ``x-amz-storage-class`` header in request metadata for determining which storage class to assign an object. +The specific syntax or method for setting headers depends on your preferred method for interfacing with the MinIO server. + +Specify the value using ``EC:M`` notation, where ``M`` refers to the number of parity blocks to create for the object. + +The following table lists the default values based on the :ref:`erasure set size ` of the initial server pool in the deployment: + +.. list-table:: + :header-rows: 1 + :widths: 30 70 + :width: 100% + + * - Erasure Set Size + - Default Parity (EC:N) -Environment Variables ---------------------- + * - 1 + - EC:0 + + * - 2-3 + - EC:1 + + * - 4-5 + - EC:2 + + * - 6 - 7 + - EC:3 + + * - 8 - 16 + - EC:4 + +The minimum supported value is ``0``, which indicates no erasure coding protections. +These deployments rely entirely on the storage controller or resource for availability / resiliency. + +The maximum value depends on the erasure set size of the initial server pool in the deployment, where the upper bound is :math:`\frac{\text{ERASURE_SET_SIZE}}{\text{2}}`. +For example, a deployment with erasure set stripe size of 16 has a maximum standard parity of 8. + +You can change this value after startup to any value between ``0`` and the upper bound for the erasure set size. +MinIO only applies the changed parity to newly written objects. +Existing objects retain the parity value in place at the time of their creation. + +Reduced Redundancy Storage Class +-------------------------------- .. note:: - These settings do not have configuration setting options for use with :mc:`mc admin config set`. + *MinIO Storage Classes* are distinct from *AWS Storage Classes*. + + AWS Storage Classes refer to the specific storage tier on which to store a given object, such as ``hot`` or ``glacier`` storage. + MinIO Storage Classes affect the erasure code parity setting used and relate to :ref:`minio-availability-resiliency` of objects. + + For tiering from one type of storage to another, such as for cost management purposes, see :ref:`minio-lifecycle-management-tiering`. + +.. tab-set:: + + .. tab-item:: Environment Variable + :sync: envvar + + .. envvar:: MINIO_STORAGE_CLASS_RRS + + .. tab-item:: Configuration Setting + :sync: config -.. envvar:: MINIO_STORAGE_CLASS_STANDARD + .. include:: /includes/common-mc-admin-config.rst + :start-after: start-minio-settings-no-config-option + :end-before: end-minio-settings-no-config-option - The :ref:`parity level ` for the deployment. - MinIO shards objects written with the default ``STANDARD`` storage class using this parity value. +The :ref:`parity level ` for objects written with the ``REDUCED`` storage class. - MinIO references the ``x-amz-storage-class`` header in request metadata for determining which storage class to assign an object. - The specific syntax or method for setting headers depends on your preferred method for interfacing with the MinIO server. +MinIO references the ``x-amz-storage-class`` header in request metadata for determining which storage class to assign an object. +The specific syntax or method for setting headers depends on your preferred method for interfacing with the MinIO server. - Specify the value using ``EC:M`` notation, where ``M`` refers to the number of parity blocks to create for the object. +Specify the value using ``EC:M`` notation, where ``M`` refers to the number of parity blocks to create for the object. - The following table lists the default values based on the :ref:`erasure set size ` of the initial server pool in the deployment: +This value **must be** less than or equal to :envvar:`MINIO_STORAGE_CLASS_STANDARD`. - .. list-table:: - :header-rows: 1 - :widths: 30 70 - :width: 100% +You cannot set this value for deployments with an erasure set size less than 2. +Defaults to ``EC:1`` for deployments with erasure set size greater than 1. +Defaults to ``EC:0`` for deployments of erasure set size of 1. - * - Erasure Set Size - - Default Parity (EC:N) +Parity Retention Optimization +----------------------------- - * - 1 - - EC:0 - * - 2-3 - - EC:1 +.. tab-set:: - * - 4-5 - - EC:2 + .. tab-item:: Environment Variable + :sync: envvar - * - 6 - 7 - - EC:3 + .. envvar:: MINIO_STORAGE_CLASS_OPTIMIZE - * - 8 - 16 - - EC:4 + .. tab-item:: Configuration Setting + :sync: config - The minimum supported value is ``0``, which indicates no erasure coding protections. - These deployments rely entirely on the storage controller or resource for availability / resiliency. - - The maximum value depends on the erasure set size of the initial server pool in the deployment, where the upper bound is :math:`\frac{\text{ERASURE_SET_SIZE}}{\text{2}}`. - For example, a deployment with erasure set stripe size of 16 has a maximum standard parity of 8. + .. mc-conf:: storage_class optimize + :delimiter: " " - You can change this value after startup to any value between ``0`` and the upper bound for the erasure set size. - MinIO only applies the changed parity to newly written objects. - Existing objects retain the parity value in place at the time of their creation. +MinIO by default automatically "upgrades" parity for an object if the destination erasure set maintains write quorum *but* has one or more drives offline. +This behavior helps ensure that the given object maintains the same availability as objects written to the healthy erasure set. -.. envvar:: MINIO_STORAGE_CLASS_RRS +Specify ``capacity`` to this setting to direct MinIO to not create any additional parity for the object. +This prioritizes the overall capacity of the cluster at the cost of potentially reduced object availability in the event more drives in that erasure set fail. - The :ref:`parity level ` for objects written with the ``REDUCED`` storage class. +Comment +------- - MinIO references the ``x-amz-storage-class`` header in request metadata for determining which storage class to assign an object. - The specific syntax or method for setting headers depends on your preferred method for interfacing with the MinIO server. +.. tab-set:: - Specify the value using ``EC:M`` notation, where ``M`` refers to the number of parity blocks to create for the object. + .. tab-item:: Environment Variable + :sync: envvar - This value **must be** less than or equal to :envvar:`MINIO_STORAGE_CLASS_STANDARD`. + .. envvar:: MINIO_STORAGE_CLASS_COMMENT - You cannot set this value for deployments with an erasure set size less than 2. - Defaults to ``EC:1`` for deployments with erasure set size greater than 1. - Defaults to ``EC:0`` for deployments of erasure set size of 1. + .. tab-item:: Configuration Setting + :sync: config -.. envvar:: MINIO_STORAGE_CLASS_COMMENT + .. include:: /includes/common-mc-admin-config.rst + :start-after: start-minio-settings-no-config-option + :end-before: end-minio-settings-no-config-option - Adds a comment to the storage class settings. +Adds a comment to the storage class settings.