Merge branch 'error-msg-bulk-insert' into ito

Update to SOCI branch showing row values for errors during bulk
operations.

See #1200

CHANGES

@@ -23,15 +23,27 @@ Other changes in v4.1.0:
 Changes affecting all or multiple backends:
  - Make rowset more convenient and safer (#198, #1057, #1081, #1082, #1086).
  - Fix problems with dynamic cast and libc++ (#913, #975).
- - Improve BLOB support consistency across backends (#991).
+ - Improve BLOB support consistency across backends (#992).
+ - Dynamic backends loading improvements (#1169).
+ - Fix several problems detected by UBSAN and use it in the CI builds too now.
+ - Improvements to connection string handling (#1176).
+ - Fix get_affected_rows() after partial updates for ODBC and SQLite (#1199).
 
 Backend-specific changes:
 
+- Oracle
+ - Fix conversion of dates before 1900 (#1135).
+ - Fix using non-ASCII characters as CLOB data (#1184).
+
 - PostgreSQL
  - Add support for network address types (#1078).
 
 - SQLite3
  - Add "nocreate" option to avoid creating a new database file (#1021).
+ - Improve column names to SOCI database types mapping (#1120).
+
+- ODBC
+ - Add support for using wide strings (#1179).
 
 
 History of the changes in the previous versions:

docs/api/backend.md

@@ -1,6 +1,6 @@
 # Backends reference
 
-This part of the documentation is provided for those who want towrite (and contribute!) their
+This part of the documentation is provided for those who want to write (and contribute!) their
 own backends. It is anyway recommendedthat authors of new backend see the code of some existing
 backend forhints on how things are really done.
 
@@ -28,6 +28,7 @@ enum data_type
 enum db_type
 {
     db_string,
+    db_wstring,
     db_int8,
     db_uint8,
     db_int16,
@@ -50,6 +51,7 @@ enum exchange_type
 {
     x_char,
     x_stdstring,
+    x_stdwstring,
     x_int8,
     x_uint8,
     x_int16,

docs/api/client.md

@@ -1,7 +1,7 @@
 # API Reference
 
 The core client interface is a set of classes and free functions declared in the `soci.h` header file.
-All names are dbeclared in the `soci` namespace.
+All names are declared in the `soci` namespace.
 
 There are also additional names declared in the `soci::details` namespace, but they are not supposed to be directly used by the users of the library and are therefore not documented here.
 When such types are used in the declarations that are part of the "public" interface, they are replaced by "IT", which means "internal type".
@@ -13,7 +13,7 @@ The following types are commonly used in the rest of the interface:
 
 ```cpp
 // data types, as seen by the user
-enum db_type { db_string, db_date, db_double, db_int8, db_uint8, db_int16, db_uint16, db_int32, db_uint32, db_int64, db_uint64 };
+enum db_type { db_string, db_wstring, db_date, db_double, db_int8, db_uint8, db_int16, db_uint16, db_int32, db_uint32, db_int64, db_uint64 };
 
 // deprecated data types enum which may be still used but is less precise than db_type
 enum data_type { dt_string, dt_date, dt_double, dt_integer, dt_long_long, dt_unsigned_long_long };
@@ -76,7 +76,11 @@ public:
     void set_log_stream(std::ostream * s);
     std::ostream * get_log_stream() const;
 
+    void log_query(std::string const & query);
+    void clear_query_parameters();
+    void add_query_parameter(std::string name, std::string value);
     std::string get_last_query() const;
+    std::string get_last_query_context() const;
 
     void uppercase_column_names(bool forceToUpper);
 

docs/backends/odbc.md

@@ -75,11 +75,14 @@ For the ODBC backend, this type mapping is:
 | SQL_INTEGER                                               | db_int32                   | int32_t                       |
 | SQL_BIGINT                                                | db_int64                   | int64_t                       |
 | SQL_CHAR, SQL_VARCHAR                                     | db_string                  | std::string                   |
+| SQL_WCHAR, SQL_WVARCHAR, SQL_WLONGVARCHAR                 | db_wstring                 | std::wstring                  |
 | SQL_TYPE_DATE, SQL_TYPE_TIME, SQL_TYPE_TIMESTAMP          | db_date                    | std::tm                       |
 
 Not all ODBC drivers support all datatypes.
 Columns having the attribute `unsigned` get mapped to their corresponding `db_uint[n]` and `uint[n]_t` types.
 
+Note that SQL Server `TINYINT` columns don't support negative values (i.e. always behave as `unsigned`), please use `SMALLINT` if you need to store small negative integer values instead.
+
 (See the [dynamic resultset binding](../types.md#dynamic-binding) documentation for general information on using the `row` class.)
 
 ### Binding by Name

docs/beyond.md

@@ -19,11 +19,6 @@ if ( !st.get_affected_rows() )
 }
 ```
 
-### Portability note
-
-This method behaviour in case of partially executed update, i.e. when some records were updated or inserted while some other have failed to be updated or inserted, depends on the exact backend and, in the case of ODBC backend, on the exact ODBC driver used.
-It can return `-1`, meaning that the number of rows is unknown, the number of rows actually updated or the total number of affected rows.
-
 ## Sequences
 
 It is common to have auto-incrementing database fields or fields whose value come from a sequence.

docs/binding.md

@@ -144,8 +144,7 @@ v.resize(100);
 sql << "select ...", into(v);
 ```
 
-In the latter case the initial size of the vector defines the maximum number of data elements that the user is willing to accept and after executing the query the vector will be automatically resized to reflect that actual number of rows that were read and transmitted.
-That is, the vector will be automatically shrunk if the amount of data that was available was smaller than requested.
+In the latter case the initial size of the vector defines the maximum number of data elements that the user is willing to accept, and so must be non-zero and, usually, relatively big for optimal performance. After executing the query the vector will be automatically resized to reflect that actual number of rows that were read and transmitted, i.e. the vector will be automatically shrunk if the amount of data that was available was smaller than requested. Note that this means that when performing a bulk operation in a loop, the vector needs to be resized during each loop iteration.
 
 It is also possible to operate on the chosen sub-range of the vector:
 

docs/logging.md

@@ -10,6 +10,7 @@ The following members of the `session` class support the basic logging functiona
 * `void set_log_stream(std::ostream * s);`
 * `std::ostream * get_log_stream() const;`
 * `std::string get_last_query() const;`
+* `std::string get_last_query_context() const;`
 
 The first two functions allow to set the user-provided output stream object for logging.
 The `NULL` value, which is the default, means that there is no logging.
@@ -26,7 +27,24 @@ An example use might be:
 Each statement logs its query string before the preparation step (whether explicit or implicit) and therefore logging is effective whether the query succeeds or not.
 Note that each prepared query is logged only once, independent on how many times it is executed.
 
-The `get_last_query` function allows to retrieve the last used query.
+The `get_last_query` function allows to retrieve the last used query. The associated `get_last_query_context` function allows to obtain a string
+representation of the bound values (if any) used in the last query. That is, while `get_last_query` might return something like
+`INSERT INTO dummy (val) VALUES (:val)`, `get_last_query_context` will return something like `:val=42` and together they can be used to get a detailed
+understanding of what happened in the last query. Cached parameters are cleared at the beginning of each query and are **not** persisted across
+multiple queries.
+
+Logging of query parameters is **enabled by default** but can at any time be adjusted to your needs by using the `set_query_context_logging_mode`
+function of the `session` object. For instance
+
+    sql.set_query_context_logging_mode(log_context::on_error);
+
+Possible values are
+
+* `log_context::always` - Always cache the bound parameters of queries (the default)
+* `log_context::never` - Never cache bound parameters. This also ensures that bound parameters are not part of exception messages. It is therefore
+  suitable for queries that bind sensitive information that must not be leaked.
+* `log_context::on_error` - Only caches bound parameters in case the query encounters an error. This is intended for cases in which you don't want to
+  have the overhead of caching parameters during regular operations but still want this extra information in case of errors.
 
 ## Flexible logging using custom loggers
 

docs/types.md

@@ -138,6 +138,55 @@ Users are encouraged to use the latter as it supports a wider range of numerical
 The mapping of underlying database column types to SOCI datatypes is database specific.
 See the [backend documentation](backends/index.md) for details.
 
+While `row::get<T>()` (semantically) copies the internally stored data (that was fetched from the database), there is also `row::move_as<T>`, which
+instead makes use of C++11's move semantics by moving the data out of the `row` instance. In case the used `T` does not support move semantics,
+`move_as` is functionally equivalent to `get`.
+
+### Dealing with Blobs
+
+If the fetched data is of type `db_blob` it is strongly recommended to use `row::move_as<blob>()` in order to obtain the data as a `blob` object. This
+allows for the most flexible and efficient access of the underlying data.
+
+Note that it is not possible to `row::get<blob>()` as `blob` objects are not copyable. However, you can instead get the blob's data into a
+`std::string` or a `std::vector<T>` as long as `T` has a size of exactly one byte (i.e. is a byte-like type such as `char` or `std::byte`). Depending
+on the used backend, this will either directly fetch the data from the database directly into the provided container or copy the already fetched data
+into the provided container.
+
+If you want to use a different container than the ones mentioned above, you need to specialize the `soci::is_contiguous_resizable_container` trait.
+E.g.
+
+```cpp
+template<>
+struct ::soci::is_contiguous_resizable_container<MyAwesomeContainer> : std::true_type {};
+
+template<typename ValueType>
+struct ::soci::is_contiguous_resizable_container<boost::container::vector<ValueType>, std::enable_if_t<sizeof(ValueType) == sizeof(char)>> : std::true_type {};
+```
+
+Potentially, you may also need to specialize the `soci::contiguous_resizable_container_accessor` struct, if the default implementation doesn't work
+for your custom container type:
+
+```cpp
+template<typename T, typename = std::enable_if_t<is_contiguous_resizable_container_v<T>>>
+struct contiguous_resizable_container_accessor
+{
+    // Gets the pointer to the beginning of the data store
+    static void *data(T &container)
+    {
+        static_assert(sizeof(decltype(container[0])) == sizeof(char), "Expected value-type of container to be byte-sized");
+        return &container[0];
+    }
+
+    // Gets the size **in bytes** of this container
+    static std::size_t size(const T &container) { return container.size(); }
+
+    // Resizes the container to the given size **in bytes**
+    static void resize(T &container, std::size_t size) { container.resize(size); }
+};
+```
+
+### Indicators
+
 The `row` also provides access to indicators for each column:
 
 ```cpp
@@ -149,6 +198,8 @@ if (r.get_indicator(0) != soci::i_null)
 }
 ```
 
+### Stream API
+
 It is also possible to extract data from the `row` object using its stream-like interface, where each extracted variable should have matching type respective to its position in the chain:
 
 ```cpp

include/private/soci-compiler.h

@@ -36,6 +36,19 @@
 #   define SOCI_GCC_WARNING_RESTORE(x)
 #endif
 
+// SOCI_MSVC_WARNING_{SUPPRESS,RESTORE} macros are similar but for MSVC (they
+// work for all the supported versions).
+#if defined(_MSC_VER)
+#   define SOCI_MSVC_WARNING_SUPPRESS(x) \
+        __pragma(warning(push)) \
+        __pragma(warning(disable:x))
+#   define SOCI_MSVC_WARNING_RESTORE(x) \
+        __pragma(warning(pop))
+#else
+#   define SOCI_MSVC_WARNING_SUPPRESS(x)
+#   define SOCI_MSVC_WARNING_RESTORE(x)
+#endif
+
 // CHECK_CXX_STD(version) evaluates to 1 if the C++ version is at least the
 // version specified.
 #if defined(_MSVC_LANG)

include/private/soci-exchange-cast.h

@@ -36,6 +36,12 @@ struct exchange_type_traits<x_stdstring>
   typedef std::string value_type;
 };
 
+template <>
+struct exchange_type_traits<x_stdwstring>
+{
+  typedef std::wstring value_type;
+};
+
 template <>
 struct exchange_type_traits<x_int8>
 {
@@ -114,7 +120,7 @@ struct exchange_type_traits<x_blob>
   typedef blob value_type;
 };
 
-// exchange_type_traits not defined for x_statement, x_rowid and x_blob here.
+// exchange_type_traits not defined for x_statement and x_rowid here.
 
 template <exchange_type e>
 typename exchange_type_traits<e>::value_type& exchange_type_cast(void *data)

include/private/soci-vector-helpers.h

@@ -33,6 +33,8 @@ inline std::size_t get_vector_size(exchange_type e, void *data)
             return exchange_vector_type_cast<x_char>(data).size();
         case x_stdstring:
             return exchange_vector_type_cast<x_stdstring>(data).size();
+        case x_stdwstring:
+            return exchange_vector_type_cast<x_stdwstring>(data).size();
         case x_int8:
             return exchange_vector_type_cast<x_int8>(data).size();
         case x_uint8:
@@ -76,6 +78,9 @@ inline void resize_vector(exchange_type e, void *data, std::size_t newSize)
         case x_stdstring:
             exchange_vector_type_cast<x_stdstring>(data).resize(newSize);
             return;
+        case x_stdwstring:
+            exchange_vector_type_cast<x_stdwstring>(data).resize(newSize);
+            return;
         case x_int8:
             exchange_vector_type_cast<x_int8>(data).resize(newSize);
             return;
@@ -131,6 +136,7 @@ inline std::string& vector_string_value(exchange_type e, void *data, std::size_t
             return exchange_vector_type_cast<x_xmltype>(data).at(ind).value;
         case x_longstring:
             return exchange_vector_type_cast<x_longstring>(data).at(ind).value;
+        case x_stdwstring:
         case x_char:
         case x_int8:
         case x_uint8:

include/soci/exchange-traits.h

@@ -155,6 +155,13 @@ struct exchange_traits<std::string>
     enum { x_type = x_stdstring };
 };
 
+template <>
+struct exchange_traits<std::wstring>
+{
+    typedef basic_type_tag type_family;
+    enum { x_type = x_stdwstring };
+};
+
 template <>
 struct exchange_traits<std::tm>
 {

include/soci/firebird/soci-firebird.h

@@ -198,6 +198,7 @@ struct SOCI_FIREBIRD_DECL firebird_statement_backend : details::statement_backen
     long long get_affected_rows() override;
     int get_number_of_rows() override;
     std::string get_parameter_name(int index) const override;
+    int get_row_to_dump() const override { return current_row_; }
 
     std::string rewrite_for_procedure_call(std::string const &query) override;
 
@@ -231,6 +232,9 @@ struct SOCI_FIREBIRD_DECL firebird_statement_backend : details::statement_backen
 
     long long rowsAffectedBulk_; // number of rows affected by the last bulk operation
 
+    // Return the number of rows affected by last operation.
+    long long get_last_row_count();
+
     virtual void exchangeData(bool gotData, int row);
     virtual void prepareSQLDA(XSQLDA ** sqldap, short size = 10);
     virtual void rewriteQuery(std::string const & query,
@@ -249,6 +253,11 @@ struct SOCI_FIREBIRD_DECL firebird_statement_backend : details::statement_backen
     std::map <std::string, int> names_;
 
     bool procedure_;
+
+private:
+    // Used during bulk operations to keep track of the row which potentially
+    // resulted in an error.
+    int current_row_ = -1;
 };
 
 struct SOCI_FIREBIRD_DECL firebird_blob_backend : details::blob_backend

include/soci/log-context.h

@@ -0,0 +1,22 @@
+//
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// http://www.boost.org/LICENSE_1_0.txt)
+//
+
+#ifndef SOCI_LOG_CONTEXT_H_INCLUDED
+#define SOCI_LOG_CONTEXT_H_INCLUDED
+
+namespace soci
+{
+
+enum class log_context
+{
+    never,
+    always,
+    on_error,
+};
+
+} // namespace soci
+
+#endif // SOCI_LOG_CONTEXT_H_INCLUDED