RU2624549C2 - Watermark signal generation and embedding watermark - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: watermark signal generator generating a watermark signal carrying the watermark data for hidden embedding into an audio signal in the reduction of the watermark signal and the audio signal. The watermark signal feeding circuit includes a psychoacoustic processor, that determines the audio signal masking threshold, and a modulator, that generates a watermark signal based on the superposition of the reference forming functions mutually spaced by the temporary time reference amount of the watermark data discrete-time representation, from which each function, forming the time reference, has an amplitude weighted by using the respective time reference of the discrete-time representation multiplied by the corresponding amplitude weight depending on the masking threshold, wherein the time reference is shorter than the time interval, for which the time reference forming functions are defined; and an individual amplitude weight depends on the time reference of the discrete-time representation bordering this time reference.

EFFECT: optimal balancing the ability to calculate the watermark signal and the lack of its effect on hearing.

12 cl, 39 dwg

Description

Technical field

The claimed invention relates to a device for generating (provider) a watermark signal for providing a watermark signal and for embedding a watermark using a watermark signal.

State of the art

In many technical applications, there is a need to introduce additional information into data or into a signal representing useful data, or “basic data”, such as, for example, an audio signal, a video signal, a graph, measurement readings, and the like. It is often required that this kind of additional information attached to the main data (in particular, audio data, video data, image data, measurement data, text information, etc.), was introduced so that it was invisible to the user To these data. Further, in some cases it is desirable that the attached information could not be removed from the main data (the same audio data, video data, image data, measurement data, and so on).

This is especially true for applications where electronic copyright protection is required. Moreover, sometimes the introduction into useful data of important additional, not perceived by the sensory organs, related information is simply necessary. Such a case is, for example, the inclusion in the audio content of service information containing information about the source of the proposed phonogram, its contents, about the rights that this audio content is burdened with, etc.

The concept of embedding additional information in useful data or in “master data” is called “watermarking”, that is, literally - “watermarking” (watermarking / watermarking). A discussion in the specialized literature on watermarking techniques involves a large number types of useful information, such as phonograms, video materials, images, texts and the like.

The following is a series of links to the publication of digital watermarking concepts. This also includes a wide range of manuals, manuals and other publications that deal in detail with electronic watermarking issues.

DE 19640814 C2 describes a coding method in which a non-sound data signal is introduced into an audio signal, and a method for decoding a data signal accompanying an audio signal in non-sound form. The coding method for introducing a non-sonic data signal into an audio signal is to convert the audio signal to a spectral representation. This coding method also includes determining the masking threshold of the audio signal and supplying a pseudo-noise signal. This encoding method also includes introducing a data signal and multiplying the pseudo-noise signal by a data signal to obtain a data signal with frequency extension. The encoding method also includes weighting the extension data signal with a masking threshold and overlapping the audio signal and a weighted data signal.

In addition, WO 93/07689 describes a method and apparatus for automatically identifying a program transmitted by a radio station or television channel or recorded on a medium by adding an inaudible encoded message to the sound of a program identifying a broadcast channel or radio station, program and / or exact date . When implemented according to the specified document, the sound signal is transmitted through an analog-to-digital converter to a processor, which provides for the possibility of crushing the frequency components and changing the energy of some frequency components by the calculated value with the formation of the encoded identification signal. The processor output is connected via a digital-to-analog converter with an audio output for transmitting an audio signal to the air or for recording a phonogram. In another embodiment, the solution to this document uses an analog bandwidth to isolate the frequency band of the audio signal, in which the energy can be changed to encode the audio signal.

No. 5,450,490 describes a device and methods for introducing a code having at least one code frequency component in an audio signal. Different frequency components of the audio signal are evaluated for the ability to mask the code frequency component for human hearing, and based on these estimates, the amplitude of the code frequency component is set. A description is also given of methods and apparatus for recognizing a code in an encoded audio signal. The code frequency component of the encoded audio signal is recognized based on the expected code amplitude or the interference amplitude in the audio frequency range, including the frequency of the encoding component.

WO 94/11989 describes a method and apparatus for encoding / decoding broadcast or recorded sound fragments and monitoring the location of their listeners. Methods and apparatus for encoding and decoding information as part of radio transmissions or as part of recorded signals of an audio fragment are described. In the circuit version described in the document, an audience monitoring system encodes identifying information as a component of an audio signal of a radio broadcast or a fragment of a phonogram using spread spectrum coding (broadband coding). The monitoring device receives an acoustically reproduced version of a radio transmission or a recorded signal through a microphone, decodes the identifying information of the component of the audio signal regardless of the presence of ambient background noise, and stores this information in memory by automatically writing a diary (journal) to this audience member, which is later downloaded to a centralized system device. Another monitoring device decodes the additional information of the broadcast signal, correlated with the information of the diary of the audience in the Central system device. This monitor can simultaneously send data to a centralized system device using a dial-up telephone line and receive data from a centralized system device via a signal encoded using spread spectrum technology and modulated by a third-party radio signal.

WO 95/27349 discloses an apparatus and methods for introducing codes into audio signals and decoding. A device and methods for introducing a code having at least one encoding frequency component in an audio signal are described. An assessment is made of the ability of different frequency components of the audio signal to mask the code frequency component for a person’s hearing, and based on the results of this evaluation, an amplitude is assigned to each of the coding frequency components. A description is also given of methods and apparatus for recognizing a code in an encoded audio signal. The code frequency component of the encoded audio signal is recognized based on the expected code amplitude or the interference amplitude in the audio frequency range, including the frequency of the encoding component.

However, when introducing watermark information into a time-frequency spectrogram of an acoustic signal, it is difficult to hide such information with a masking threshold or find the optimal ratio between the maximum allowable energy of the embedded watermark data necessary to ensure it is extractable ™ on the decoder side and keeping it outside audibility when playing back a watermarked audio signal.

SUMMARY OF THE INVENTION

In this situation, the purpose of the present invention is to propose a watermark signal generating circuit and a watermark embedding mechanism using a watermark signal, which would optimally balance the possibility of isolating the watermark signal and the absence of its impact on hearing.

This goal is achieved through the use of a watermark signal shaper according to paragraph 1 of the claims, a watermarking device according to paragraph 8, methods according to claim 9 or 10, and a computer program according to claim 11.

In the present invention, a watermark signal generator is provided for generating a watermark signal that is covertly inserted into an audio signal and carries watermark data; structurally incorporating a psychoacoustic processor, which estimates the threshold for masking the audio signal; and a modulator that generates a watermark signal based on a superposition of the functions forming the samples spaced apart by the time count of the discrete-time representation of the watermark data, and the amplitude of each function forming the count is weighted by multiplying the corresponding sample of the discrete-time representation by corresponding amplitude weight depending on the masking threshold; moreover, the modulator is characterized in that the value of the time reference is shorter than the time extension of the functions of forming the samples; and the corresponding amplitude weight also depends on the samples of the discrete-time representation, bordering the corresponding time sample.

The idea of this invention is that the optimal ratio between extractability and the absence of a hearing effect of the watermark signal can be achieved by selecting amplitude weights for the sampling functions that constitute a watermark signal in a superposition, not only depending on the masking threshold, but also in depending on the samples of the discrete-time representation of the watermark data adjacent to the corresponding sample. Due to this, the functions of forming adjacent samples can overlap each other, that is, the time discrete can be shorter than the time extension of the function forming the counting, because, despite this, the mutual influence of such adjacent forming forming samples can be compensated for by discrete-time discrete representations adjacent to the currently weighed sample, when setting the weight of the amplitude. Moreover, since the sampling functions allow greater time expansion, their frequency characteristics can be narrowed, so that the extractability of the watermark signal can be adjusted in intensity depending on the reverb, that is, when the watermarked audio signal is reproduced in a reverberant medium. In other words, the dependence of the individual amplitude weight not only on the masking threshold, but also on samples of the discrete-time representation of watermark data adjacent to the corresponding sample, allows you to compensate for the audible interference of adjacent functions of the formation of samples, which otherwise could lead to a violation of the masking threshold .

Brief Description of the Drawings

Further, technical solutions in accordance with the invention will be described with reference to the accompanying drawings, in which Fig. 1 is a schematic block diagram of a watermark input device according to the invention; figure 2 is a schematic block diagram of a watermark decoder according to the invention; figure 3 is given a more detailed block diagram of the watermark generator according to the invention; figure 4 is a detailed block diagram of the modulator as an element of the circuitry of the invention; figure 5 is a block diagram of the working cycle of the psychoacoustic processor as an element of the circuitry of the invention; Fig.6 is a block diagram of the operating cycle of the processor of the psychoacoustic model as an element of the circuitry of the invention; 7 shows a graph of the dependence of the spectral power density of the audio signal at the output of block 801 on frequency; in Fig.8 shows a graph of the spectral density of the power of the audio signal at the output of block 802 on frequency; figure 9 is a block diagram of the calculation of the amplitude; on figa shows a schematic diagram of a modulator; 10b is a graphical representation of an example distribution of coefficients on a time-frequency plane; on figa and 11b in the schematic block diagrams presents an alternative structural solution of the synchronization module Fig.12a graphically displays the task of temporary alignment of the watermark; 12b graphically depicts the task of identifying the start of a message; FIG. 12d graphically illustrates the construction of a time correspondence of synchronization sequences in the synchronization mode of an incomplete message; on f) graphically presents the input data of the synchronization module; 12f graphically illustrates the concept of identifying a synchronization point; Fig. 12g shows a schematic block diagram of a correlator of a synchronization signature (match marking); FIG. 13a graphically illustrates an example of a time convolution; 13b graphically illustrates an example of bitwise multiplication of bits and extension sequences; on Fig with a graph of the signal at the output of the correlator of the synchronization signature after averaging over time; on fig.13d presents a graph of the signal at the output of the correlator of the synchronization signature after filtering using the autocorrelation function of the synchronization signature; Fig. 14 is a schematic block diagram of a structural solution for a watermark extractor in accordance with the invention; Fig. 15 schematically displays a selection of a portion of a representation in the time-frequency domain as a candidate message; on Fig shows a schematic block diagram of an analysis module; in FIG. The spectrogram of the output signal of the synchronization correlator is shown; on fig.17b decoded messages are schematically displayed; on Fig with graphically displays the synchronization position isolated from the signal with a watermark; on figa graphically presents useful information, useful information with a finite Viterbi sequence, useful information in Viterbi coding and payload in Viterbi coding in a version encoded with repetitions; on fig.18b graphically displays the subcarrier frequencies used to introduce a watermarked signal; 19 is a graphical representation of an uncoded message, an encoded message, synchronization messages, and a watermark signal in which a synchronization sequence is applied to messages; on Fig schematically shows the first step of the algorithm of the so-called "ABC-synchronization"; Fig.21 graphically displays the second step of the algorithm of the so-called "ABC-synchronization"; on Fig graphically displays the third step of the so-called "ABC synchronization" algorithm; on Fig shows a message diagram containing useful information and a CRC component; on Fig shows a block diagram of the implementation of the watermark signal feeder according to the invention; and 25 is a schematic block diagram of an implementation of a watermark marking device according to the present invention.

Detailed technical description

1. Watermarking

Next, a description is given of a watermark signal feeder 2400 with reference to FIG. A psychoacoustic processor 2410 and a modulator 2420 are included in the watermark signal feeder circuit 2400. The psychoacoustic processor 2410 provides for receiving an audio signal 2430, for which a watermark signal generator 2400 must generate a watermark signal 2440. The modulator 2420, in turn, provides for the use of a masking threshold derived a psychoacoustic processor 2410 to generate a watermark signal 2440. The modulator 2420 is characterized in that it generates a watermark signal 2440 based on a superposition of functions the formation of samples spaced one from the other by the time count of the discrete-time representation of the watermark data 2450 displayed in the watermark signal 2440. In particular, when generating a watermark signal 2440, the modulator 2420 uses a masking threshold so that the watermark signal 2440 when introduced into the acoustic signal 2430 could be hidden in the output audio signal, marked with a watermark.

According to a more detailed characterization given below, a discrete-time representation of watermark data may in fact be a discrete time-frequency representation derived from watermark data 2450 using an extension in the time domain and / or frequency domain. A time or frequency-time grid with the coordinates of the samples of the discrete time representation can be set in time and, most importantly, regardless of the audio signal 2430. In turn, the superposition can be interpreted as a convolution of the discrete time representation, the samples of which are ordered on the above coordinate grid and have weighted amplitudes, which, in turn, depend not only on the masking threshold, but also on time-adjacent samples of the discrete-time representation.

The dependence of the amplitude weights on the masking threshold can be as follows: the amplitude weight, which must be multiplied by a certain sample of the discrete-time representation in a certain time block, is derived from the corresponding time block of the masking threshold, which, in turn, is itself in the time-frequency dependence. Thus, with a discrete time-frequency representation of watermark data, each sample is multiplied by the weight of the amplitude, which corresponds to the masking threshold selected at the corresponding position of the time-frequency lattice of this sample in the watermark representation structure. In addition, you can use time-differential coding (time difference coding), which makes it possible to extract a discrete time representation from 2450 watermark data. The features of design solutions are discussed in more detail below.

Modulator 2420 is characterized by the ability to generate a watermark signal 2440 based on a superposition of the sample forming functions, so that the amplitude of each sample forming function is weighted by multiplying the corresponding sample of the discrete-time representation by the corresponding amplitude weight depending on the masking threshold determined by the psychoacoustic processor 2410. In particular, the modulator 2420 is characterized in that the time discrete value is shorter than the time extension of the form Bani frame, and in that the weight corresponding to the amplitude is also dependent on neighboring data samples increments discrete-time representation.

As described in more detail below, as a result of the fact that the time count is shorter than the time extension of the counting functions, interference arises between adjacent counting functions, which increases the likelihood of spontaneous overcoming of the masking threshold. Nevertheless, such a violation of the masking threshold is compensated by the simultaneous dependence of the amplitude weights on adjacent samples of the discrete-temporal representation.

Below we consider a constructive solution of the system of "applying" electronic watermarks, where the aforementioned dependence is implemented through iterative regulation of amplitude weights. For this, the psychoacoustic processor 2410 is configured to set the masking threshold independently of the watermark data, while the modulator 2420 is configured to iteratively amplitude weights, having previously set the amplitude weights based on the masking threshold, regardless of the watermark data. the 2420 modulator controls whether the masking threshold is violated by a superposition of sample-forming functions having an amplitude, weighted by multiplying the watermark representation samples by the predetermined amplitude weights. If violated, the 2420 modulator provides for the variation of the preset amplitude weights with the formation of a different superposition. involves multiple iteration with control, variation and updating of the superposition until certain ovy stop, such as limit values iterated variable weights amplitudes. Due to the fact that with the above control, neighboring samples of the discrete temporal representation mutually influence / interfere due to the superposition and temporal expansion of the functions of forming the samples that go beyond the time frame, the entire iterative process during generation depends on these neighboring samples of the representation of watermark data.

We draw attention to the fact that in the design solutions discussed below, the extension of the watermark data in the time domain is applied to highlight the discrete temporal representation indicated above. At the same time, such a temporary expansion may be omitted. The same applies to the frequency extension, also applied in the implementations discussed below.

2. Watermarking device

On Fig presents a watermarking device that uses the output of the signal feeder watermark 2400 in Fig.24. The watermarking device 2500 in FIG. 25 includes, in addition to the watermark signal feeder 2400, an adder 2510 for converting the watermark signal 2440 coming from the watermark signal feeder 2400 and the audio signal 2430 to form an output watermarked audio signal 2530.

3. System Description

Next, a watermark transmission system that includes a watermark input unit and a watermark decoder will be described. Of course, the watermark input unit and the watermark decoder can be used independently of each other.

The principle “from complex to simple” was chosen to describe the system. First, a differentiation was made between the encoder and decoder. Then, in sections 3.1 through 3.5, each working unit is described in detail.

The basic structure of the system is shown in figures 1 and 2, where, respectively, the encoder side and the decoder side are displayed. 1 shows a schematic block diagram of a watermark input unit 100. On the encoder side, a processing unit 101 (designated as a watermark generator) generates a watermark signal 101b from binary data 101a and from information exchange 104, 105 with psychoacoustic processor 102. Information received from block 102 is intended to guarantee inaudibility of the watermark. Then, the watermark generated by the watermark generator 101 is added to the audio signal 106. Thereafter, the watermarked signal 107 can be relayed, stored, or transmitted for further processing. Multimedia files, such as audio and video files, require a significant delay in the video stream to avoid loss of audio and video synchronization. In the case of a multi-channel audio signal, each channel is processed separately, according to the explanation given in this document. Processing units 101 (watermark generator) and 102 (psychoacoustic processor) are discussed in detail in sections 3.1 and 3.2, respectively.

2, a decoder side is shown in a block diagram of a watermark detector 200. System 200 receives a watermarked audio signal 200a, for example, from a microphone. The first block 203, designated as an analysis module, demodulates and transforms the data (for example, a watermarked audio signal) in the time / frequency domain (thereby forming a time-frequency representation 204 of the audio signal with a watermark 200a), sending them to the synchronization module 201, which analyzes the input signal 204 and performs synchronization, in particular, calculates the temporal alignment of the encoded data (e.g., encoded watermark data relative to the representation in the time-frequency domain). This information (for example, the resulting synchronization data 205) is fed to the watermark extractor 202, which decodes the received data (generating the corresponding binary data 202a, which represent the data content of the watermarked audio signal 200a).

3.1 Watermark Generator 101

A detailed diagram of the watermark generator 101 is shown in FIG. 3. Binary data (expressed as ± 1) that must be hidden in the audio signal 106 is input to the watermark generator 101. Block 301 orders the received data 101a into packets of the same length M _p . Service bits are added to each packet as signal (for example, in the form of a postfix). Let M _s denote their number. Their use will be explained in more detail in section 3.5. In the future, each packet of bits of useful information together with the attached service signal bits will be called a message.

Each message 301a of length N _m = M _s + M _{p is} sent to a channel encoder, processing unit 302, which is responsible for encoding bits to protect against errors. Perhaps a constructive solution to this module, in which it consists of a convolutional encoder in combination with a device for temporary compression of pulse signals. The conversion coefficient of the convolutional encoder has a great influence on the overall degree of error protection of the digital watermarking system. On the other hand, a device for temporary compression of pulse signals provides protection against bursts of noise. The operating range of a temporary pulse signal compaction device may be limited to one message, but it may be expanded to a larger number of messages. Denote the coding coefficient, for example, 1/4, as R _c . The number of encoded bits for each message will be N _m / R _c . Channel encoder 302 generates, for example, an encoded binary message 302a.

The next processing unit 303 performs the extension in the frequency domain. To ensure a satisfactory signal-to-noise ratio, information (e.g., binary message 302a) is expanded and transmitted over N _{f to} carefully selected subbands. Their exact frequency positioning is determined a priori and set for both the encoder and the decoder. Details of the selection of this important system parameter are given in Section 3.2.2. The frequency extension is determined by the extension sequence c _{f of} dimension N _f × l. The output signal 303a of block 303 consists of N _f binary streams — one for each subband. The binary stream (bitstream) i is calculated by multiplying the input bit by the component i of the extension sequence c _f . The simplest extension is to copy the bitstream to each output stream, in particular, to use an extension sequence of units.

Block 304, designated as a synchronization circuit input device, adds a synchronization signal to the bitstream. Reliable synchronization is important because the decoder lacks parameters for temporal alignment of both bits and data structure, i.e., data about the beginning of each message. The clock signal consists of N _s sequences of N _f with N _f bits in each. Sequences are elementwise and periodically multiplied by the stream (or streams) of bits 303a. Suppose a, b, and c make up Ns = 3 synchronization sequences (denoted as synchronization extension sequences).

Block 304 multiplies a by the first propagation bit, b by the second extension bit, and c by the third extension bit. For subsequent bits, iteration is applied, namely, a is multiplied by the fourth bit, b by the fifth, and so on. Accordingly, the integrated information about the synchronization data 304a is generated. Synchronization sequences (designated as extension synchronization sequences) are carefully selected to minimize the risk of false synchronization. Further details are provided in section 3.4. In addition, it should be borne in mind that the sequence a, b, c, ... can be considered as a sequence of synchronizing extension sequences.

Block 305 performs expansion in the time domain. Each extension bit at the input, in particular, a vector of length N _f , is repeated in the time domain N _t times. Similar to the frequency expansion, we calculate the expansion sequence with _t dimension Nt × 1. The repetition of i time is multiplied by the i-th component c _t .

Mathematically, the operations of blocks 302-305 can be expressed as follows. Let m dimension 1 × N _m = R _c be the encoded message at the output of block 302.

Then, at the output of block 303, signal 303a (which can be considered as a representation of the extension data R) will have the form

,

,

at the output of block 304, signal 304a, which can be considered as a combined representation of data synchronization c, has the form

,

,

where ° denotes the elementary work of Schur, and

.

.

The output signal 305a of block 305 is

,

,

where ◊ and ^T denote Kronecker multiplication and transposition, respectively. Remember that binary data is expressed as ± 1.

Block 306 performs differential bit coding. This step gives the system additional stability against phase shifts caused by the motion or mismatch of the local oscillator. This issue is discussed in more detail in section 3.3. If b (i; j) is a bit for frequency band i and time block j at the input of block 306, the output bit b _diff (i; j) will be

.

.

At the beginning of the stream, that is, for j = 0, b _diff (i, j-1) is set to 1.

Block 307 performs the actual modulation, that is, the generation of the waveform of the watermark signal depending on the binary input data 306a. Details of the circuit solution [modulator 307] are given in figure 4. N _f parallel inputs from 401 to 40 N _f contain bitstream for different subbands. Each bit of each sub-band stream is processed by a bit generation unit (411-41N _f ). The output of the bit formation blocks are waveforms in the time domain. The waveform generated for time block j and subband i, denoted as S _{i; j} (t), based on the input bit b _diff (i, j) is calculated as follows

, косинусно модулированной по частотеwhere y (i; j) is the weight coefficient calculated by the psychoacoustic processor 102, T _b is the bit interval, and g _i (t) is the function that forms the bit for subband i. The bit-forming function is derived from the baseband function

cosine modulated in frequency

,

,

where f _i is the average frequency of subband i, and the superscript T denotes the transmitter. Baseband functions may be different for each subband. With out identity on the decoder side, a more efficient implementation is possible. See section 3.3 for a more detailed description.

Each bit is generated in an iterative mode controlled by a psychoacoustic processor (102). Iterations are necessary to fine-tune the balance γ (i, j) to give the watermark the maximum possible energy while maintaining its inexperience. Further details are provided in section 3.2.

The full waveform at the output of the i-th bit formation filter 41i is

.

.

обычно не равна нулю для интервала времени, намного превышающего T_b, притом, что основная энергия сконцентрирована в интервале бита. В качестве примера можно привести график на фигуре 12а, где одна и та же формирующая бит основополосная функция применена для двух смежных битов. На фигуре T_b=40 мс. Выбор T_b, как и форма функции значительно влияет на систему. Фактически, большее количество символов дает более узкие частотные характеристики. Это особенно благоприятно для реверберирующих сред. Действительно, при подобных сценариях сигнал с водяным знаком попадает в микрофон по нескольким путям распространения, каждый из которых характеризуется отличным от других временем прохождения. Результирующий канал характеризуется значительной частотной избирательностью. При интерпретации во временной области более длинные символы имеют то преимущество, что они как эхо имеют задержку, сопоставимую с усиливающей интерференцией, результирующей из битового интервала, что означает, что они наращивают энергию принимаемого сигнала. Несмотря на это, более длинные символы имеют ряд недостатков: большее перекрывание может вызывать межсимвольную интерференцию (ISI), и его гораздо труднее скрыть в аудиосигнале таким образом, чтобы психоакустический процессор пропускал меньшую энергию, чем для более коротких символов.Bit forming baseband function

usually not equal to zero for a time interval far exceeding T _b , while the main energy is concentrated in the bit interval. As an example, we can cite the graph in figure 12a, where the same bit-forming baseband function is applied to two adjacent bits. In the figure, T _b = 40 ms. The choice of T _b , as well as the form of the function, significantly affects the system. In fact, a larger number of characters gives a narrower frequency response. This is especially beneficial for reverberant environments. Indeed, in such scenarios, a watermarked signal enters the microphone through several propagation paths, each of which has a different travel time. The resulting channel is characterized by significant frequency selectivity. When interpreted in the time domain, longer characters have the advantage that they, as an echo, have a delay comparable to the amplifying interference resulting from the bit interval, which means that they increase the energy of the received signal. Despite this, longer characters have several drawbacks: greater overlap can cause intersymbol interference (ISI), and it is much harder to hide in the audio signal so that the psychoacoustic processor passes less energy than for shorter characters.

The watermark signal is output as the total output of all bit formation filters

.

.

3.2 Psychoacoustic processor 102

As can be seen in figure 5, the psychoacoustic processor 102 consists of 3 elements. The first element is an analysis module 501, which converts a temporary audio signal into a time / frequency domain. This analysis module can perform analysis in parallel with different time / frequency resolutions. From the analysis module, the time / frequency data [504] goes to the psychoacoustic modeling unit (RAM) 502, which calculates the masking thresholds for the watermark using psychoacoustic parameters (see E.Zwicker H. Fasti, “PsychoacousticsFactsandModels"). Masking thresholds serve as an indicator of the amount of energy which may be hidden in the audio signal for each subband and time block.The last element of the psycho-acoustic processor 102 is the amplitude calculation module 503. This module calculates the amplification levels by amplitude, which should be are named when generating a watermark signal with observance of masking thresholds, that is, the energy involved must be less than or equal to the energy given by the masking thresholds.

3.2.1 Time / Frequency Analysis 501

Block 501 performs time / frequency conversion of the audio signal by converting with overlap. The best sound quality can be achieved by using multilevel time / frequency resolution. An efficient overlapping transform is the short (/ window) Fourier transform (FFT) (FFT), based on the fast Fourier transform (FFT) of window-weighted time blocks. The length of the window determines the time-frequency resolution in such a way that longer windows are characterized by a longer time and higher frequency resolution, while short windows are vice versa. However, the shape of the window, among other things, determines the frequency leakage.

In the proposed system, we achieve an inaudible watermark by analyzing the data through two different levels of resolution. The first filter bank is characterized by a step size T _b , i.e. - bit length. The step size is the time interval between two adjacent time blocks. The window length is approximately equal to T _b . It should be borne in mind that the shape of the window does not have to repeat the shape of the window used to form the bit, but should generally mimic the human hearing system. The study of this problem is the subject of many publications.

The second filter bank uses a shorter window. High temporal resolution is especially important when watermarking speech, since its temporal structure is usually thinner than T _b .

The sampling frequency of the input audio signal is not important, since it is sufficient to describe a watermark signal without aliasing. For example, if the largest frequency component of the watermark signal is 6 kHz, then the sampling frequency of the temporary signals should be at least 12 kHz.

3.2.2 Block of psychoacoustic modeling 502

The function of the psychoacoustic modeling unit 502 is to determine masking thresholds, i.e., to calculate the energy that can be hidden in the audio signal for each frequency subband and time block so that the watermarked audio signal remains indistinguishable from the original sound background.

. Подполосы определяют путем выделения N_f средних частот f_i и допущения, что

. Надлежащий выбор средних частот обеспечивает шкала Барка, предложенная Цвикером в 1961 году. Для более высоких средних частот подполосы увеличиваются. В одной из возможных версий реализации системы использовано 9 поддиапазонов, расположенных соответствующим образом между 1,5 и 6 кГц.Subband i is defined between two limits, namely,

. The subbands are determined by allocating N _f mid frequencies f _i and assuming that

. The proper midrange selection is provided by the Bark scale proposed by Zwicker in 1961. For higher mid frequencies, the subbands increase. In one possible version of the implementation of the system, 9 subbands are used, located respectively between 1.5 and 6 kHz.

The following processing operations are performed separately for each level of the time-frequency resolution for each frequency subband and each time block. At process stage 801, spectral smoothing is performed. Both tonal elements and gaps in the power spectrum should be smoothed. This can be done in several ways. The measure of tonality can be calculated for the further selection of an adaptive smoothing filter. In a simplified alternative embodiment of this block, a median filter may be used. The median filter takes into account the vector of indicators and displays their median (median) value. In the median-like filter, a value corresponding to a quantile other than 50% can be selected. The filter width is determined in Hz and is used as a non-linear moving average that starts at low frequencies and ends at the highest possible frequency. The operation [spectrum smoothing] 801 is illustrated in figure 7. The red curve (dashed line) indicates the resulting smoothing signal.

After smoothing is performed using block 802, only frequency masking thresholds are calculated. This procedure also has embodiments. One way is to use a minimum for each subband component to calculate the masking energy E _i . This is energy equivalent to a signal that effectively provides masking. Based on this value, simply by multiplying by a certain scale factor, we can obtain the masked energy J _i . Such coefficients will be different for each subband and time-frequency resolution; they are derived empirically based on psychoacoustic experiments. Fragments of such a scenario are illustrated in figure 8.

Using block 805, threshold temporal masking values are calculated. In this case, different time blocks are analyzed for the same subband component. The masked energies j _{i are} modified in accordance with empirically derived post-masking profiles. Consider two adjacent time blocks, k-1 and k. The corresponding masked energies are J _i (k-1) and J _i (k). The post-masking profile determines, for example, that the masking energy E _i is able to mask the energy J _i during k and α⋅J _i during k + 1. In this case, block 805 compares J _i (k) (the energy masked by the current time block) and α⋅J _i (k + 1) (the energy masked by the previous time block) and selects the maximum. Post-masking profiles are built empirically, based on psychoacoustic experiments, and are available in the relevant literature. It should be noted that for large values of T _b , i.e. > 20 ms, post-masking is applied only to the time-frequency resolution of shorter time windows.

So, at the output of block 805, we have masking thresholds for each subband component and time block, calculated for two different time-frequency resolutions. These threshold values are obtained taking into account both frequency and temporary masking phenomena. Using block 806, threshold values of various time-frequency resolutions are combined. For example, one of the possible options is one in which block 806 takes into account all threshold values corresponding to the time-frequency intervals in which the bit is located and selects a minimum.

3.2.3 Amplitude calculation block 503

Turning to figure 9. The input data of block 503 are the output threshold values 505 of the block of psychoacoustic modeling 502, which performs all the calculations associated with psychoacoustics. The amplitude calculator 503 performs additional calculations using the threshold values entered therein. In the first block, amplitude mapping 901 is performed. Using this block, the values of masking thresholds (usually expressed as energy values) are converted into amplitudes that can be used to scale the bit formation function described in Section 3.1. Next, the amplitude adaptation block 902 is activated. This block is designed to iteratively adapt the amplitudes γ (i, j) used to multiply the bit-forming functions in the watermark generator 101 in such a way as to in fact satisfy the masking thresholds. As already mentioned, in fact, the function of forming a bit, as a rule, takes a period of time in excess of T _b . Therefore, multiplying the correct amplitude γ (i, j) that satisfies the masking threshold at i, j does not necessarily satisfy the requirements at i, j-1. This is especially important in cases of attacks when a pre-echo is acoustically manifested. Another situation that should be avoided is an inappropriate superposition of the tails of different bits, resulting in a voiced watermark. Thus, block 902 analyzes the output of the watermark generator against threshold values. In the event of a mismatch, this block modifies the amplitudes γ (i, j) appropriately.

This completes the consideration of the encoder side. The following sections are devoted to the technological operations of the receiver (designated as a watermark decoder).

3.3 Analysis module 203

204, по одному потоку для каждой подполосы i спектра. Затем, битовые потоки поступают для обработки в модуль синхронизации 201 и в экстрактор водяного знака 202, что описано в разделах 3.4 и 3.5, соответственно. Следует обратить внимание, что

представляют собой гибкие двоичные потоки, что значит, что они могут, например, принимать любое действительное значение без окончательного оформления бита.Analysis module 203 is the first step (or block) for watermark extraction. Its purpose is in the inverse transformation of the audio signal with a watermark 200a to N _f bitstream

204, one stream for each subband i of the spectrum. Then, the bit streams are sent for processing to the synchronization module 201 and to the watermark extractor 202, as described in sections 3.4 and 3.5, respectively. It should be noted that

They are flexible binary streams, which means that they can, for example, take any real value without finalizing the bit.

The analysis module consists of three components shown in FIG. 16: analysis filter bank 1600, amplitude normalization block 1604, and differential decoding block 1608.

3.3.1 Analysis filter bank 1600

для ветви или подполосы i в момент времени j. Эти значения содержат информацию об амплитуде и фазе сигнала на средней частоте f_i и в момент времени j⋅Tb.Using an analyzing filter bank 1600, a detailed diagram of which is shown in FIG. 10a, a watermarked audio signal is converted into a time-frequency domain. At the input, the filter bank receives an audio signal with a watermark r (t). At the output, it generates complex coefficients

for branch or subband i at time j. These values contain information about the amplitude and phase of the signal at the middle frequency f _i and at time j⋅Tb.

Filter bank 1600 consists of N _f branches, one for each subband i. Each branch is divided into an upper branch for the in-phase component and a lower branch for the quadrature component of subband i. Despite the fact that the modulation in the watermark generator and, therefore, the watermarked audio signal have simple real values, a comprehensive analysis of the signal is necessary on the receiver side, since the receiver does not have data on the alternation of modulation factors due to the influence of channels and synchronization mismatches. Consider further the branch of 1 filter bank. Combining in-phase and quadrature branches, we calculate a complex-valued modulating signal

,

,

- импульсная характеристика фильтра нижних частот приемника для подполосы i.Как правило,

i (t) равно функции формирования бита базовой полосы пропускания подполосы i в модуляторе 307, чтобы выполнялись условия согласованного фильтра, однако возможны и другие импульсные характеристики.where * denotes a convolution, a

- impulse response of the receiver low pass filter for subband i. As a rule,

i (t) is equal to the bit generation function of the base bandwidth of subband i in modulator 307 so that the conditions of the matched filter are satisfied, however, other impulse responses are possible.

при скорости 1-T_b, необходим отбор отсчетов непрерывного выходного сигнала

. Если бы на приемной стороне была известна точная синхронизация битов, дискретизация со скоростью 1=T_b была бы достаточной. Однако, поскольку данные синхронизации битов отсутствуют, дискретизацию выполняют со скоростью N_os/T_b, где N_os - коэффициент избыточности дискретизации банка фильтров анализа. Подбирая достаточно большое значение N_os (например, N_os=4), можно гарантировать, что, по меньшей мере, один цикл дискретизации достаточно близок к идеальной синхронизации битов. Оптимальный уровень дискретизации с повышенной частотой выбирается в процессе синхронизации, поэтому все данные избыточной дискретизации сохраняются до начала этой операции. Подробнее этот процесс описан в разделе 3.4.To get the odds

at a speed of 1-T _b , sampling of the continuous output signal is necessary

. If the exact timing of the bits were known at the receiver side, sampling at a rate of 1 = T _b would be sufficient. However, since there is no bit synchronization data, sampling is performed at a rate of N _os / T _b , where N _os is the sampling redundancy coefficient of the analysis filter bank. Choosing a sufficiently large value of N _os (for example, N _os = 4), it can be guaranteed that at least one sampling cycle is close enough to perfect bit synchronization. The optimal sampling level with increased frequency is selected during the synchronization process, therefore, all data of oversampling is stored until the beginning of this operation. This process is described in more detail in section 3.4.

, где j обозначает номер бита или момент времени, а к обозначает позицию уплотненной дискретизации внутри этого одиночного бита, где k=1; 2;…N_os.At the output of the branch we obtain the coefficients

where j denotes the bit number or point in time, and k denotes the position of the compressed sampling inside this single bit, where k = 1; 2; ... N _os .

.Figure 10b graphically shows an example of the distribution of the coefficients on the time-frequency plane. The sampling redundancy coefficient N _os = 2. The height and width of the rectangles indicate, respectively, the bandwidth and length of time of the signal fragment represented by the corresponding coefficient

.

If the frequencies of the subband f _i are selected as multiples of a certain interval Δf, the analysis filter bank can be effectively implemented using the fast Fourier transform (FFT).

3.3.2 Amplitude Normalization 1604

на входе блока нормализации 1604. В силу того, что информация о состоянии канала недоступна на приемной стороне (т.е. канал прохождения не известен), применяют схему линейного суммирования сигналов равной мощности (EGC). Вследствие рассеивающей способности канала по времени и частоте энергия переданного бита b_i(j) обнаруживается не только вокруг средней частоты £и момента времени j, но и на соседних частотах и в смежные моменты времени. Поэтому для более точного взвешивания на частотах f_i±nΔf рассчитывают дополнительные коэффициенты, которые используют для нормализации коэффициента

). Если n=1. мы имеем, например,In order to simplify the description without departing from the main topic, in the future we assume that the synchronization of bits is known, and that N _os = 1. This means that we have complex coefficients

at the input of normalization block 1604. Due to the fact that information about the channel state is not available at the receiving side (i.e., the transmission channel is not known), a linear summation circuit of equal power signals (EGC) is used. Due to the channel’s scattering ability in time and frequency, the energy of the transmitted bit b _i (j) is detected not only around the average frequency £ and time j, but also at neighboring frequencies and at adjacent times. Therefore, for more accurate weighing at frequencies f _i ± nΔf, additional coefficients are calculated, which are used to normalize the coefficient

) If n = 1. we have for example

Normalization for n> 1 is a direct extension of the above formula. In the same way, we can choose flexible bits to normalize, given more than one point in time. Normalization is performed for each subband i and each time j. The actual addition of the EGC is performed in the later steps of the extraction process.

3.3.3 Differential decoding 1608

амплитуд, содержащие информацию о фазе компонент сигнала на частоте f_i в момент времени j. Поскольку биты дифференциально закодированы на передающей стороне, здесь должно быть произведено обратное действие. Гибкие биты

формируются путем, во-первых, вычисления разности фаз двух последовательных коэффициентов и, затем, выделения вещественной части:The input of the differential decoding unit 1608 receives the normalized complex coefficients

amplitudes containing information about the phase of the signal components at a frequency f _i at time j. Since the bits are differentially encoded on the transmitting side, the opposite must be done here. Flexible bits

formed by, firstly, calculating the phase difference of two successive coefficients and, then, highlighting the material part:

These actions must be performed separately for each subband, since the channel usually introduces different phase rotations in each subband component.

3.4 Synchronization module 201

, использованные при синтезе в модуляторе, должны быть приведены в соответствие с фильтрами

используемыми для анализа. Эта задача проиллюстрирована на фигуре 12а, где фильтры анализа идентичны фильтрам синтеза. На верхнем графике показаны три бита. Для упрощения формы волны для всех трех битов не масштабированы. Шаг по времени между битами - T_b. На нижнем графике проиллюстрирована процедура синхронизации в декодере: фильтр может быть применен в разные моменты времени, тем не менее, правильными являются только координаты по красной (жирной) кривой 1299а, которые обеспечивают экстракцию первого бита с лучшими показателями соотношения сигнал-шум SNR и соотношения сигнал-помеха SIR. На практике некорректное выравнивание привело бы к деградации SNR и SIR. Мы обозначили это первое выравнивания термином „синхронизация битов". Благодаря достижению синхронизации битов извлечение битов выполняется в оптимальном режиме. При этом, чтобы корректно расшифровать сообщение, необходимо знать, с какого бита начинается новое сообщение. Эта задача, называемая синхронизацией сообщения, графически проиллюстрирована на фигуре 12b. В потоке декодированных битов только начальная позиция 1299b, маркированная красным, является верной и позволяет декодировать сообщение к.The task of the synchronization module is to temporarily align the watermark. Synchronizing a decoder with respect to encoded data is a twofold task. At the first stage, the filter bank of the analysis should be brought into line with the encoded data, in particular, the function of the formation of bits

used in the synthesis in the modulator must be aligned with the filters

used for analysis. This task is illustrated in figure 12A, where the analysis filters are identical to the synthesis filters. The upper graph shows three bits. For simplicity, the waveforms for all three bits are not scaled. The time step between bits is T _b . The lower graph illustrates the synchronization procedure in the decoder: the filter can be applied at different points in time, however, only the coordinates on the red (bold) curve 1299a are correct, which provide extraction of the first bit with the best SNR and signal-to-signal ratios Interference SIR. In practice, incorrect alignment would lead to degradation of SNR and SIR. We designated this first alignment with the term “bit synchronization.” Due to the achievement of bit synchronization, the extraction of bits is performed in the optimal mode. In order to correctly decrypt the message, you need to know which bit the new message starts from. This task, called message synchronization, is graphically illustrated on figure 12b. In the stream of decoded bits, only the initial position 1299b, marked in red, is correct and allows you to decode the message to.

First, we turn only to message synchronization. The synchronization signature, as explained in section 3.1, consists of a predefined order of sequences that are continuously and periodically injected into the watermark. The synchronization module is configured to recognize temporal alignment of synchronization sequences. Depending on the size N, two modes of operation are distinguished, which are displayed in figures 12c and 12d, respectively.

In the synchronization mode of the complete message (Fig. 12 c) N _s = N _m / R _c . To simplify the figure, it is assumed that N _s = N _m / R _c = 6 and N _t = 1, i.e. no time extension. For clarity, the used synchronization signature is shown below the messages. In fact, they are modulated depending on the encoded bits and frequency extension sequences, as explained in section 3.1. In this mode, the frequency of the synchronization signature matches the frequency of the messages. Thus, the synchronization module can recognize the beginning of each message by temporarily aligning the synchronization signature. The time coordinates of the start of the new synchronization signature are indicated as synchronization points. Then, the synchronization points are transmitted to the watermark extractor 202.

The second possible mode - incomplete message synchronization is illustrated in figure 12d. In this case, N _s <N _m = R _c . For this figure, it is assumed that N _s = 3, so that the three synchronization sequences are repeated twice for each message. Keep in mind that the frequency of messages is not necessarily a multiple of the frequency of the synchronization signature. In this mode, not all synchronization points coincide with the beginning of the message. The synchronization module does not provide for determining the difference between the synchronization points, and this operation is assigned to the watermark extractor 202.

The processing units of the synchronization module are shown in figures 11a and 11b. The synchronization module simultaneously performs bit synchronization and message synchronization (complete or incomplete) by analyzing the output signal of the correlator of the synchronization signature 1201. Data during the time / frequency domain 204 comes from the analysis module. Since bit synchronization is not yet available, block 203 resamples the data with a factor of N _os , as described in section 3.3. Full input is shown in Figure 12e. In this example, N _os = 4, N _t = 2, and N _s = 3. In other words, the synchronization signature consists of 3 sequences (indicated by a, b, and c). The time extension, in this case, using the extension sequence c _t = [1 1] ^T , is performed by simply repeating each bit twice in the time domain. The exact synchronization points are indicated by arrows and correspond to the beginning of each synchronization signature. The synchronization signature period is N _t ⋅ N _os ⋅ N _s = N _sbl , which, for example, is 2⋅4⋅3 = 24. Due to the periodicity of the synchronization signature, the synchronization signature correlator (1201) arbitrarily divides the time axis into blocks called search blocks of size N _Sbl , where the subscript indicates the length of the search block. Each search block should contain (or usually contain) one synchronization point, as shown in FIG. 12f. Each of the N _Sbl bits is a candidate synchronization point. The task of block 1201 is to calculate the likelihood degree for each candidate bit in each block. Further, this information is transmitted to block 1204 for calculating synchronization points.

3.4.1 Sync Signal Correlator 1201

For each synchronization candidate position N _{sbl, the} synchronization signature correlator calculates a likelihood index that is the higher, the higher the likelihood that time alignment (both bit and full or incomplete message synchronization) is recognized. The sequence of technological operations shown in figure 12g.

As a result, a sequence 1201a of likelihood indicators associated with various selected positions can be derived.

Block 1301 performs a temporary convolution, that is, multiplies each of the bits N _t by a sequence of temporal expansion and then sums them. This action is performed for each of the frequency subbands N _f . Figure 13a shows an example. For example, the same parameters are taken as in the previous section, namely, N _os = 4, N _t = 2, and N _s = 3. Candidate sync position marked. From this bit in increments of N _os, block 1301 counts N _t ⋅ N _s and performs time convolution using the sequence c _t , leaving bits Ns.

At block 1302, these bits are multiplied elementwise by the extension sequence N _s (see FIG. 13b).

Block 1303 performs frequency convolution, namely, multiplies each bit by the extension sequence c _f and then sums over the frequency.

At this stage, if the synchronization position is correct, we would have decoded bits N _s . Since the bits are not known on the receiver side, block 1304 calculates a measure of likelihood using absolute values of the values and sums N _s .

Basically, the output of block 1304 needs an incoherent correlator that is looking for a synchronization signature. In practice, when choosing a small value of N _s , namely, the synchronization mode of an incomplete message, you can use synchronization sequences (for example, a, b, c) that are mutually orthogonal. Moreover, if the correlator is incorrectly aligned with the signature, its output value will be very small, ideally equal to zero. If the full message synchronization mode is activated, it is recommended to use as many orthogonal synchronization sequences as possible, and then create a signature by carefully selecting the order of their application. In this case, the same theory can be applied as when searching for an extension sequence with the proper autocorrelation functions. When the correlator has only a slight mismatch, the output value of the correlator will not be equal to zero even in the ideal case, but in any case it will be less than the exact alignment, since the analysis filters cannot optimally capture the signal energy.

3.4.2 Calculation of synchronization points 1204

This block analyzes the output of the synchronization signature correlator to determine synchronization positions. Due to the fact that the system is quite resistant to mismatches up to T _b / 4, and T _b usually approaches 40 ms, the output of block 1201 can be integrated over time to achieve more stable synchronization. The implementation of this approach is possible using a filter with an infinite impulse response (IIR) in time with an exponentially decaying impulse response. Alternatively, a conventional filter with a moving average value of the final impulse response (FIR) can be applied. Since the averaging has already been performed, a second correlation is performed between the other N _t выполняютN _s ("select a different position"). In fact, we want to use information where the autocorrelation function of the synchronization function is known. This corresponds to the maximum likelihood estimation. This aspect is illustrated in Fig. 13c The curve displays the output signal of block 1201 after temporary integration.One possible way to determine the point of synchronism is to simply find the maximum of this function. In figure 13d we see the same function (lean curve) after filtering by the autocorrelation function of the synchronization signature.The graph of the resulting function is highlighted in red (bold) .In this case, the maximum is more pronounced and determines the coordinates of the synchronization point. These two methods are approximately the same in terms of high SNR, but the second method is better suited for lower SNR modes: Once the synchronization points are found, they are introduced into the watermark extractor 202, which decodes the data.

In some versions of the execution for the formation of a stable clock signal, synchronization is performed in the mode of incomplete communication with short synchronization signatures. This requires a large amount of decoding, which increases the risk of recognition of false messages. To prevent this phenomenon in some implementations, signal sequences are inserted into messages, which leads to a lower bit rate.

This approach is applicable in solving the problem that occurs when the synchronization signature is shorter than the message, which was already discussed above in the context of synchronization optimization. In this case, the decoder does not know where the new message begins, and he tries to start decoding at several synchronization points. To distinguish between valid and false-valid messages, some technical solutions involve the use of a signal word (i.e., useful information is sacrificed in favor of including a known control sequence). In some embodiments, a likelihood control (alternatively or additionally) is used to differentiate recognition of valid and false messages.

3.5 Watermark Extractor 202

The layout elements of the watermark extractor 202 are shown in Figure 14. It has two inputs, namely, 204 and 205, from blocks 203 and 201, respectively. From the synchronization module 201 (see section 3.4), synchronizing time stamps are received, that is, coordinates in the time domain where the candidate message begins. This issue is discussed in more detail in section 3.4. The second input from the bank of analyzing filters 203 receives time / frequency domain data, ready for decoding.

In a first processing step, a data selector 1501 recognizes a component of the input data 204 as an intended message for decoding. In figure 15b, this procedure is presented graphically. Input 204 consists of N _f streams of real values. Since the decoder is not known a priori time alignment data, the analysis unit 203 performs frequency analysis at a speed of more than 1 / T _b Hz (oversampling). In the context of FIG. 15b, a sampling redundancy factor of 4 is used, namely, every vector vectors of dimension N _f × 1 are generated every T _b seconds. When the synchronization unit 201 identifies the candidate message, it issues a time stamp 205 indicating the reference point of the intended message. The data selection unit 1501 selects the information necessary for decoding, namely, a matrix of dimension N _f × N _m / R _c . This matrix 1501a is transmitted to block 1502 for subsequent processing.

Blocks 1502, 1503, and 1504 perform the same operations as blocks 1301, 1302, and 1303, as described in section 3.4.

An alternative technical solution of the invention excludes from the calculation algorithm using blocks 1502-1504 and complements the functions of the synchronization module by preparing data for decoding. From a conceptual point of view, this is a particular. The point of view of technical performance is just a matter of how buffers are implemented. In general, re-performing calculations reduces the size of buffers.

The channel decoder 1505 performs the inverse operation to block 302. If the channel encoder in a possible embodiment of this module consists of a convolutional encoder in combination with a device for temporarily compressing pulse signals, the channel decoder will perform decompression and convolutional decoding, for example, using the well-known Viterbi algorithm . At the output of this block, we get N _m bits, i.e. - candidate message.

The signaling and likelihood control unit 1506 determines whether the input candidate message is a valid message. There are various algorithms for solving this problem.

The basic idea is to use a signal word (for example, a CRC cyclic redundancy check sequence) to differentiate between true and false messages. However, this reduces the number of bits for useful information. Alternatively, you can use likelihood control. For example, if messages contain a timestamp, consecutive messages must contain consecutive timestamps. If the decoded message has a time stamp that does not correspond to the specified order, such a message may be excluded.

After correctly recognizing the message, the system can select a lead and / or return mechanism. It is understood that synchronization of both bits and messages has been achieved. Assuming that the user intentionally does not miss messages, the system performs a “return” in time and attempts to decode previous messages (if they have not already been decoded) using the same synchronization point (“return” algorithm). This is especially useful when starting up the system. In addition, under adverse conditions, two messages may be required to achieve synchronization. In this case, the first message has no chance. Thanks to the “return” algorithm, it is possible to save “benign” messages that were not received only due to the use of reverse synchronization. Anticipation is the opposite of this function with a future reference. If there is a message at the moment, we know where the next message should be, and in any case, we can try to decode it.

3.6. Detail synchronization

For coding useful information, in particular, the Viterbi algorithm can be applied. On figa shows the payload 1810, the final Viterbi sequence 1820, the Viterbi encoded payload 1830, and the Viterbi payload coding option with repeated coding 1840. So, let the payload length be 34 bits, and the final Viterbi sequence consists of 6 bits. If, suppose Viterbi coding rate 1/7 is applicable, Viterbi encoded useful information may contain (34 + 6) * 7 = 280 bits. Further, when using 1/2 repetition coding, the repetitively encoded version 1840 of Viterbi-encoded payload 1830 may contain 280 * 2 = 560 bits. In this example, if we assume that the bit cycle is 42.66 ms, the message length will be 23.9 seconds. The signal can be introduced using, for example, 9 subcarriers (for example, localized in accordance with the critical frequency bands) from 1.5 to 6 kHz, as shown in the frequency spectrum diagram in Fig. 18b. Alternatively, a different number of subcarrier frequencies (say, 4, 6, 12, 15, or between 2 and 20) in the frequency range between 0 and 20 kHz can be used.

FIG. 19 schematically illustrates a basic synchronization concept 1900, also called ABC synchronization. The diagram graphically displays an unencoded message 1910, an encoded message 1920, and a synchronization sequence (sync sequence) 1930, as well as applying synchronization to several messages 1920 following each other.

The synchronization sequence or sync sequence mentioned in the context of explaining the concept of synchronization (shown in FIGS. 19 to 23) may be identical to the synchronization signature discussed earlier. Next, in FIG. 20, synchronization is shown schematically by correlation with a sync sequence. If the synchronization sequence 1930 is shorter than the message, more than one synchronization point 1940 (or temporary alignment block) can be found within a single message. In the example of FIG. 20, 4 synchronization points are defined within each message. Therefore, with each synchronization found, a Viterbi decoder (Viterbi decoding sequence) can be used. Thus, for each synchronization point 1940, a message 2110 can be received, as shown in FIG. From these messages, true messages 2210 can be extracted using a cyclic redundancy check sequence (CRC sequence) and / or likelihood control, as shown in FIG. When monitoring using a cyclic redundancy code (when recognizing CRC), a known sequence can be used to differentiate between true and false messages. On Fig graphically shows an example of attaching a sequence of CRC to the end of the useful information. The probability of false detection (messages generated based on an erroneous synchronization point) may depend on the length of the CRC sequence and the number of Viterbi decoders used (the number of synchronization points in the structure of one message). To increase the length of useful information without increasing the likelihood of false detection, you can apply likelihood control or increase the length of the synchronization sequence (synchronization signatures).

4. Approaches and benefits

Next, we consider some aspects of the system described above as signs of novelty. In addition, we compare these aspects with the current level of technology in this field.

4.1. Continuous sync

Some design solutions provide for continuous synchronization. The synchronization signal (sync signal), designated as the synchronization signature, is introduced continuously and in parallel with the data by multiplying with sequences designated as synchronization extension sequences, which are known both on the transmitting and receiving sides. Some traditional systems use special characters (different from those used for data), while such special characters are not used in implementations of the claimed invention. Other classic methods are to introduce a known sequence of bits (preambles) with time multiplexing relative to data, or to introduce a signal with frequency multiplexing relative to data.

However, it was found that the use of dedicated subbands for synchronization is undesirable, since the channel may have dips at these frequencies, which makes synchronization unreliable. Compared to other methods in which a preamble or special signal is multiplexed with data over time, the advantage of the method described here is that it allows you to track changes in synchronization (for example, due to movement) continuously.

Moreover, the energy of the watermark signal does not change (for example, when a watermark is introduced by multiplying the watermark signal into the extension information), and synchronization can be designed independently of the psychoacoustic model and data rate. The temporal extent of the synchronization signature, which determines the stability of synchronization, can be designed as desired completely independently of the data exchange rate.

Another classic method is to introduce a code multiplexing synchronization sequence with respect to data. Compared with this classical method, the advantage of the proposed method is that the energy of the data is not an interfering factor in calculating the correlation, providing greater robustness. In addition, when using code multiplexing, the number of orthogonal sequences to be synchronized is reduced since some of them are necessary for the data.

Based on the foregoing, the continuous synchronization approach described here provides a number of advantages over conventional algorithms.

However, in some hardware versions of the present invention, some other synchronization concept is applied.

4.2. 2D expansion

Some technical solutions of the proposed system perform expansion both in the time and in the frequency domain, that is, two-dimensional expansion (in short - 2D extension). It has been established that it provides advantages over 1D systems, since the transmission frequency of erroneous bits can be further reduced by introducing redundancy, for example, in the time domain.

Meanwhile, some implementations of the claimed invention allow the use of any other extension method.

4.3. Differential coding and differential decoding

In some implementations, resistance to movement and frequency mismatch of local oscillators (in contrast to traditional systems) is provided by differential modulation. It is confirmed that the Doppler effect (motion / displacement) and frequency mismatches lead to the rotation of the constellation (stereometric relative positioning) of the binary (/ on-off) phase manipulation of the DPSK (VZRD) in other words, to rotation in the complex bit plane). In some implementations, the harmful effects of such rotation of the DPSK constellation (or the constellation of any other appropriate modulation) are prevented by using differential (differential) encoding or differential decoding.

At the same time, some versions of the implementation of the present invention allow the involvement of any other encoding algorithm or decoding algorithm. Moreover, in some cases, differential coding may be excluded.

4.4. Bit shaping

Some technical solutions of the claimed invention provide a significant improvement in system performance and recognition reliability through the use of a bit formation filter. According to such technical solutions, the use of bit formation increases the reliability of watermarking. It has been determined that particularly positive results can be obtained if the bit formation function has a longer length than the bit interval. However, in some embodiments, a different bit formation concept may be used. Moreover, in some cases, bit formation may not be performed.

4.5. Interactive synthesis of the block of psychoacoustic modeling (RAM) and filter bank (FB)

Some hardware versions provide for the interaction of the block of psychoacoustic modeling with a modulator for fine tuning the amplitudes that multiply bits. At the same time, in other versions of the design, such interaction can be excluded.

4.6. Lead and Return Functions

Some technical solutions provide the so-called "return" and "lead" functions. The following is a brief overview of these concepts. If the message is decoded correctly, it means that the synchronization was successful. Assuming that the user does not clear the memory, some hardware versions return in time and attempt to decode previous messages (if they have not yet been decoded) using the same synchronization point (the “look back” function). This is especially useful when starting the system In the worst situations, 2 messages may be required to achieve synchronization. In such a case, in traditional systems, the first message has no chance. In the layout options of the invention with the return option, it provides It is possible to save (or decode) “good” messages that were not received solely due to reverse synchronization. Anticipation is the opposite of this function with a future reference. If I now have a message, then I know where my next message should be and I can try to decode it in some way. Thus, overlapping messages allows you to decode them. However, when implementing the invention in some cases, the lead function and / or return function may be excluded.

4.7. Improving synchronization reliability

In some versions of execution, in order to form a stable clock signal, synchronization is performed in the mode of incomplete communication with short synchronization signatures. This requires a large amount of decoding, which increases the risk of recognition of false messages. To prevent this phenomenon in some implementations, signal sequences are inserted into messages, which leads to a lower bit rate. However, when implementing the invention, an alternative approach is possible to optimize the reliability of synchronization. Moreover, in some cases, you can not apply any methodology to improve the reliability of synchronization.

4.8. Other enhancements

Let's move on to a selective review of a number of possible improvements to the system described above in comparison with the prior art:

1. lower computational complexity;

2. improved quality of perception due to improved psychoacoustic model;

3. increased error tolerance in reverberant environments due to narrow-band multi-carrier signals;

4. an exception in some implementations of estimating the signal-to-noise ratio (SNR). Ensuring due to this better stability, especially in low SNR modes.

Some embodiments of the invention are superior to conventional systems that use very narrow bands, for example, 8 Hz, for the following reasons:

1. an 8 Hz band (or similar very narrow frequency band) requires very long time symbols, since very little energy is allocated in the psychoacoustic model to ensure its inaudibility;

2. 8 Hz (or a similar, very narrow band) becomes noticeable within the framework of the dependence on the time-varying spectra of Doppler frequencies. Because of this, such a narrow-band system usually does not completely meet the technical requirements of the implementation, for example, in hours.

Some technical solutions of the invention are ahead of other technologies for the following reasons:

1. Echo-introducing technologies fail in reverberating spatial volumes. In contrast, the constructive solutions according to the invention are aimed at eliminating the possibility of introducing an echo.

2. Technologies that use only the time extension contain a longer message than the designs of the system described above, in which a two-dimensional extension is applied, for example, in time and frequency simultaneously.

The design solutions of the present invention are superior to the system described in DE196 40 814, since they have overcome one or more of the disadvantages listed below:

- excessive complexity of the system decoder according to DE 19640814, where a 2N filter with N = 128 is applied;

- excessive system message length according to DE 19640814,

- expansion in the system according to DE 19640814 is performed only in the time domain at a relatively high increment (for example, 128),

- excessive system complexity according to DE 19640814, where the signal is generated in the time domain, converted to the spectral region, weighted, converted back to the time domain and superimposed on the audio content.

5. Appointment

The present invention includes a method for modifying an audio signal to conceal digital data and a corresponding decoder configured to search and retrieve such data while maintaining the modified signal with original acoustic quality.

The following are examples of possible areas of application of the invention:

1. Broadcast monitoring: a watermark containing information, for example, about the station and time, is hidden in the audio signal of a radio or television program. Decoders embedded in small devices assigned to test subjects recognize a watermark and thus collect valuable information for advertising agencies, in particular, who watched which program and when.

2. Audit: a watermark may be hidden, for example, in advertisements. By automatically monitoring the broadcasts of a particular station, you can subsequently find out when exactly the advertisement was broadcast. In a similar way, you can select statistics on the broadcasting of various stations, for example, the frequency of sounding on the air of a particular piece of music, etc.

3. The inclusion of metadata: the proposed method provides the ability to enter hidden digital data of a musical work or program, for example, the name and author of the play or the duration of the program, etc.

, отстоящих друг от друга на величину временного отсчета T_b дискретно-временного представления b_diff(i, j) данных водяного знака, а именно - на равную длину M_p вышеупомянутых пакетов, причем, амплитуда каждой функции

формирования отсчета взвешена посредством умножения соответствующего отсчета b_diff(i, j) дискретно-временного представления на соответствующий амплитудный вес γ(i; j) в зависимости от порога маскирования, при этом, модулятор характеризуется тем, что величина временного отсчета T_b короче, чем временное расширение функций формирования отсчетов, как проиллюстрировано на фиг.12а; а соответствующий амплитудный вес γ(i; j) зависит также от отсчетов дискретно-временного представления, граничащих с соответствующим отсчетом повремени.So, the above circuit solutions in figure 1 - 23 and figure 24 and 25 describe: the driver of the watermark signal 2400, forming the output signal of the watermark 2440; 101b for covert insertion into the audio signal 2430; 106 and displaying watermark data 2450; 101a; at the same time, the watermark signal shaper includes a psychoacoustic processor 2410 in its circuit; 102 defining an audio masking threshold; and modulator 2420; 307 to 101, generating a watermark signal based on a superposition (Equation 8 and FIG. 12a), for example, counting functions

spaced apart by the value of the time reference T _{b of the} discrete-time representation b _diff (i, j) of the watermark data, namely, by the equal length M _{p of the} aforementioned packets, and the amplitude of each function

the formation of the sample is weighted by multiplying the corresponding sample b _diff (i, j) of the discrete-time representation by the corresponding amplitude weight γ (i; j) depending on the masking threshold, while the modulator is characterized in that the time sample T _{b is} shorter than temporal expansion of the functions of the formation of samples, as illustrated in figa; and the corresponding amplitude weight γ (i; j) also depends on the samples of the discrete-time representation adjacent to the corresponding time sample.

The psychoacoustic processor provides for the determination of the masking threshold regardless of the watermark data of 2450; the modulator, in turn, provides for iterative generation of the watermark signal, in which it first determines the amplitude weight γ (i; j) based on the masking threshold, regardless of the watermark data, and then controls whether the masking threshold is violated by a superposition of sample-forming functions using a pre-selected amplitude weight. If the threshold is overcome, the preselected amplitude weight is changed to achieve a superposition of the sample-forming functions whose modified amplitude weight will be appropriate. As explained above, since from the moment of control during iteration, adjacent samples of the discrete-time representation mutually influence / interfere due to superposition and exceeding by time expansion of the functions that form the samples, the sampling step, the entire iterative process of generating a 2440 watermark signal and the resulting acceptable amplitude the weights, respectively, depend on these adjacent watermark data presentation samples. In other words, control during iteration activates the dependence of the applied amplitude weights γ (i; j) on the samples b _diff (i, j ± 1) and ensures the proper balance between articulation and the absence of auditory effects of the watermark signal. Naturally, the iterative computational process of checking, forming a superposition, and varying weights can be repeated many times.

Otherwise, the above-described dependence on the boundary samples of the representation of the watermark data can be realized by non-iterative selection of the amplitude weights. In particular, the modulator can determine the amplitude weights γ (i; j) analytically, proceeding simultaneously from the masking threshold at (i, j) and from neighboring watermark samples b _diff (i, j ± 1).

The time expander 305 is configured to expand the watermark data in a time domain with deriving a discrete time representation. In addition, the frequency expander 303 is configured to expand the watermark data in the frequency domain with the derivation of a discrete time representation. In this case, the time / frequency analyzer 501 provides for the possibility of transferring the audio signal from the time domain to the frequency domain by converting with an overlay using the length of the first window, approximately equal to the value of the time reference. The time / frequency analyzer also provides for the possibility of transferring the audio signal from the time domain to the frequency domain by converting overlay using the length of the second window less than the length of the first window.

, входящие в суперпозицию соответствующей дискретизированной по времени частотной подполосы, содержат несущую частоту на средней частоте f_i соответствующей дискретно-временной подполосы i.Moreover, if the discrete-time representation is formed from time-sampled frequency subbands, the modulator provides for the possibility of generating a watermark signal for each discrete-time subband according to equations 8 and 9 based on the superposition of the sample-forming functions that are mutually separated by the time discrete, the amplitude of each of which is weighted by multiplying the corresponding sample of the corresponding discrete-time subband by the corresponding amplitude weight depending on pores og masking, while the function of forming samples

included in the superposition of the corresponding time-sampled frequency subband, contain the carrier frequency at the middle frequency f _{i of the} corresponding discrete-time subband i.

The following describes structural solutions for a watermarking device 2500; 100, including in its circuit a watermark signal generator 2400 and an adder 2510 for converting the watermark signal and the audio signal to output an audio signal marked with a watermark.

6. Alternative design solutions

Despite the fact that the equipment is mainly considered here from the point of view of its technical structure, it is clear that aspects of the material part are closely related to the description of the corresponding methods of its application, and any product or unit corresponds to the particularities of the method or technological operation.

Similarly, the technologies and operations under consideration are directly related to the corresponding machinery and its elemental base.

Some or all of the steps of the proposed method can be performed using hardware, such as, for example, a microprocessor, programmable computer, or electronic circuit. In some cases, the implementation of one or more critical operations that make up this method can be performed by such a device.

An encoded watermark signal or an audio signal in which a watermark signal is embedded can be stored in a digital storage device or can be transmitted to a broadcast medium such as wireless broadcasting or wired communication, such as the Internet.

Depending on the final destination and the features of practical application, the invention can be implemented in hardware or software.

In the implementation, such digital storage media as floppy disk, DVD, Blue Ray, CD, ROM, EPROM, programmable ROM, EPROM or FLASH memory containing electronically readable control signals that interact (or are compatible) can be used with a programmable computer system so that the proposed method can be implemented.Therefore, the digital storage medium may be computer readable.

Some design options according to this invention incorporate a storage medium containing electronically readable control signals compatible with a programmable computer system and capable of participating in the implementation of one of the methods described herein.

In General, this invention can be implemented as a computer program product with a program code that provides for the implementation of one of the proposed methods, provided that the computer program product is used using a computer. The program code may, for example, be stored on a computer-readable medium.

Various embodiments include a computer program stored on a computer-readable medium for implementing one of the methods described herein.

Thus, formulating differently, the method related to the invention is carried out using a computer program having a program code for implementing one of the methods described here, if the computer program is executed using a computer.

Further, therefore, the technical implementation of the invented method includes a storage medium (either a digital storage medium or a computer-readable medium) containing a computer program recorded thereon for implementing one of the methods described herein.

It follows that the implementation of the invention implies the presence of a data stream or sequence of signals representing a computer program for implementing one of the methods described here. A data stream or a sequence of signals can be designed to be transmitted via communication means, for example, the Internet.

In addition, the implementation includes hardware, for example, a computer or programmable logic device, designed or adapted to implement one of the methods described here.

Further, for technical execution, a computer with a computer program installed on it is required to implement one of the methods described here.

Some versions of the design to implement one or all of the functionality of the methods described here may require the use of a programmable logic device (for example, a field programmable matrix of logic elements). Depending on the purpose of the version, the base matrix crystal may be combined with a microprocessor to implement one of the methods described here. Typically, the described methods can be implemented using any hardware.

The structural solutions described above are only illustrations of the basic principles of the present invention. It is understood that for specialists in this field, the possibility of making changes and improvements to the layout and elements of the described construction is obvious. Because of this, the descriptions and explanations presented here of embodiments of the invention are limited only by the scope of patent requirements, and not specific details.

Claims (12)

1. A watermark signal generator (2400) that generates a watermark signal (2440; 101b) for covertly introducing the watermark data displayed by it (2430; 106) into the audio signal (2450; 101a) when mixing the watermark signal with an acoustic signal, including psychoacoustic a processor (2410; 102) that performs the calculation of the masking threshold of the audio signal, and a modulator (2420; 307) that generates a watermark signal based on a superposition of the functions that form the samples, mutually separated by the value of the time sample (T _b ) of the discrete-time representation watermarks having an individual amplitude, weighted by multiplying the corresponding sample of the discrete-time representation by the corresponding amplitude weight depending on the masking threshold, characterized in that the value of the time sample is shorter than the time interval on which the functions of forming the samples are determined; and the fact that the corresponding amplitude weight also depends on the samples of the discrete-time representation adjacent to the corresponding time sample. 2. The watermark signal generator according to claim 1, wherein the psychoacoustic processor determines a masking threshold regardless of the watermark data and comprising a modulator generates a watermark signal using iteration by preliminary calculating the expected amplitude weight based on the masking threshold, regardless of the data a watermark, by controlling whether the masking threshold is violated by a superposition of counting-forming functions having a predetermined amplitude weight; and in case of violation of the masking threshold by a superposition of counting functions with a predetermined amplitude weight - by varying a predefined amplitude weight to achieve such a superposition of sampling functions in which an appropriate individual amplitude weight will be selected for these functions. 3. The watermark signal generator according to claim 1 further includes in its construction a time expander (305), intended to expand the watermark data in the time domain with the formation of a discrete-time representation. 4. The watermark signal generator according to claim 1 further includes in its design a frequency expander (303), designed to expand the watermark data in the frequency domain with the formation of a discrete-time representation. 5. The watermark signal generator according to claim 1, wherein the psychoacoustic processor includes a time / frequency analyzer (501) in its circuitry, which converts the audio signal from the time domain to the frequency domain by converting using an overlay using the length of the first window approximately equal to the time count . 6. The watermark signal generator according to claim 5, wherein the time / frequency analyzer also translates the audio signal from the time domain to the frequency domain by converting overlay using a second window length shorter than the length of the first window. 7. The watermark signal generator according to claim 1, characterized in that when a discrete temporal representation is formed from time-sampled frequency subbands, the modulator generates a watermark signal for each discrete-time subband based on the superposition of the sample-forming functions mutually spaced by time discrete, the amplitude of each of which is weighted by multiplying the corresponding sample of the corresponding discrete-time subband by the corresponding amplitude weight in the head depending on the masking threshold, while the sampling functions included in the superposition of the corresponding time-sampled frequency subband contain the carrier frequency at the mid-frequency of the corresponding discrete-time subband. 8. A watermarking device, comprising in its design a watermark signal generator according to any one of the preceding paragraphs, generating a watermark signal for covertly introducing the watermark data it displays into the audio signal when the watermark signal is combined with an acoustic signal, and an adder performing the signal reduction watermark and audio signal with the formation of an audio signal marked with a watermark. 9. A method for generating a watermark (101b) representing watermark data (101a) covertly input to an audio signal (106) by combining a watermark signal and an audio signal, including determining a masking threshold of the audio signal; and generating a watermark signal based on a superposition of the sample-forming functions mutually spaced by the time reference value (T _b ) of the discrete-time representation of the watermark data, each of which has an amplitude weighted by a corresponding sample of the discrete-time representation multiplied by the corresponding amplitude weight depending on the masking threshold, while the generation is carried out in such a way that the time count is shorter than the time interval on which counting functions; and the corresponding amplitude weight depends simultaneously on the samples of the discrete-time representation adjacent to the corresponding time sample. 10. A method of embedding a watermark, comprising generating a watermark signal carrying watermark data for covert embedding into an audio signal by combining a watermark signal and an audio signal according to claim 9, and mixing the watermark signal and the audio signal to form an audio signal marked with a watermark. 11. A computer-readable storage medium with a computer program recorded thereon containing control commands for implementing the method of claim 9, provided that the program is executed on computer technology. 12. A computer-readable storage medium with a computer program recorded thereon containing control commands for implementing the method of claim 10, provided that the program is executed on computer technology.

Images