Writing translations

In the backend, a translation is created with the localize command of the DataHandler: it links the new record to its default language record and fills the translation fields l10n_parent, l10n_source, l10n_diffsource and l10n_state. Each scenario below writes the same translation once through the DataHandler and once through Extbase, with the means Extbase code has for it, and compares the resulting rows.

Provenance

Tests ran on main (15.0-dev), which is assumed to behave as TYPO3 14.3.

Summary

9 scenarios, by Extbase status:

Comparison

ScenarioCoreExtbase
Creating a translation links it to the default record Core bugThe English article 1 is localized to German through DataHandler, the backend action "localize". DataHandler creates the German article 2 and links it: l10n_parent and l10n_source point to article 1, l10n_diffsource holds the values of article 1 at the time of the translation, and l10n_state marks the synchronized field subtitle as following its parent. DeviatesExtbase has no localize action. The twin creates a new object, sets its language to German through the property "_languageUid" and persists it with add() and persistAll(). Extbase writes the German article 2 with the language 1, but l10n_parent and l10n_source stay 0, l10n_diffsource stays empty and l10n_state stays NULL: the new row is not linked to article 1.
A translation copies the values of the default record The English article 1 is localized to German through DataHandler, the backend action "localize". DataHandler copies the values of the title and the subtitle from article 1 to the new German article 2. Same as coreExtbase has no localize action and copies nothing by itself. The twin reads article 1 through the repository, creates a new object with the language German (property "_languageUid"), assigns the title and the subtitle of article 1 in code and persists it with add() and persistAll(). The German article 2 has the same values as with DataHandler, because the code assigned them; the row is not linked to article 1, which this scenario does not compare.
Prefix lang title prefixes the translated value The English article 1 is localized to German through DataHandler, the backend action "localize". The field label has the language prefix (prefixLangTitle), so DataHandler writes the label of the German article 2 with the prefix "[Translate to German:]" in front of the value of article 1. DeviatesExtbase has no localize action. The twin creates a new German object (property "_languageUid") with the label of article 1 and persists it with add() and persistAll(). Extbase stores the label exactly as the code assigned it, without the language prefix; the TCA setting prefixLangTitle is ignored.
Changing a translation keeps its link The title of the German translation 2 is changed through DataHandler. DataHandler keeps l10n_parent 1, writes the new title, updates l10n_diffsource with the values of the default record 1 and writes l10n_state with the synchronized field subtitle marked as following its parent. DeviatesExtbase reads the German translation through the repository (language aspect German), changes the title and persists it with update() and persistAll(). Extbase writes the new title into the translation 2 and keeps l10n_parent 1, but leaves l10n_diffsource empty and l10n_state NULL.
A synchronized field follows the default record The subtitle of the English article 1 is changed through DataHandler. The field subtitle allows language synchronization and the translation 2 follows its parent, so DataHandler writes the new subtitle into the translation as well and records the state "parent" for the field in l10n_state. DeviatesExtbase reads article 1 through the repository, changes the subtitle and persists it with update() and persistAll(). Extbase writes the new subtitle into article 1 only; the translation 2 keeps its old subtitle and l10n_state stays NULL, because Extbase does not know language synchronization.
A record for all languages A new article with the language "all languages" (-1) is created through DataHandler. DataHandler writes the row with sys_language_uid -1 and l10n_parent 0. Same as coreExtbase creates a new object, sets its language to -1 through the property "_languageUid" and persists it with add() and persistAll(). Extbase writes sys_language_uid -1 and l10n_parent 0, like DataHandler.
Saving the default tour gives the translation its own copies The title of the English tour 1 is changed through DataHandler and its stops are sent as the backend form sends them: the field stops holds "1,2". The German translation 2 has no copies yet. DataHandler creates the stops 3 and 4 as copies of the stops 1 and 2: they belong to the German tour 2, carry t3_origuid 1 and 2 and the sorting 1 and 2, and DataHandler writes the number of stops, 2, into the field stops of tour 2. DeviatesExtbase reads tour 1 through the repository, changes the title and persists it with update() and persistAll(). Extbase writes the new title into tour 1, but does not touch the German translation: it creates no copies of the stops, the stops 1 and 2 stay the only stops and the field stops of tour 2 stays 0.
Saving the default tour with two translations copies only for the last one Core bugThe title of the English tour 1 is changed through DataHandler and its stops are sent as the backend form sends them: the field stops holds "1,2". The German translation 2 and the French translation 3 have no copies yet. DataHandler creates only one set of copies, the stops 3 and 4 with t3_origuid 1 and 2, and attaches it to the French tour 3. The German tour 2 gets no copy, although the field stops of tour 2 holds 2, like that of tour 3. This pins a bug in DataHandler, see https://forge.typo3.org/issues/111069. DeviatesExtbase reads tour 1 through the repository, changes the title and persists it with update() and persistAll(). Extbase writes the new title into tour 1, but does not touch the translations: it creates no copies of the stops for the German tour 2 or the French tour 3, the stops 1 and 2 stay the only stops and the field stops of both translations stays 0.
A stop added to the default tour is copied without its origin Core bugA new stop is sent as a child of the English tour 1 through DataHandler, as the backend form sends it: the field stops holds "1,NEW1" and the stop is a NEW record with its pid, its title and the uid of the tour in its field tour. The German translation 2 already has the stop 2 as copy of stop 1. DataHandler creates the stop 3 for tour 1 and a copy of it, the stop 4, for tour 2, and writes the number of stops, 2, into both tours. The stop 4 has t3_origuid 0 instead of 3, because DataHandler duplicates the new record in the data map instead of copying it. This pins a bug in DataHandler, see https://forge.typo3.org/issues/111070. DeviatesExtbase attaches a new stop to tour 1 and persists it with update() and persistAll(). Extbase creates the stop 3 for tour 1 with the sorting 0 and writes the number of stops, 2, into tour 1. It does not touch the German translation: tour 2 gets no copy of the new stop, and its field stops stays 1.

Scenarios

A translation copies the values of the default record

Same as core

In 14.3

Core: The English article 1 is localized to German through DataHandler, the backend action "localize". DataHandler copies the values of the title and the subtitle from article 1 to the new German article 2.

Extbase: Extbase has no localize action and copies nothing by itself. The twin reads article 1 through the repository, creates a new object with the language German (property "_languageUid"), assigns the title and the subtitle of article 1 in code and persists it with add() and persistAll(). The German article 2 has the same values as with DataHandler, because the code assigned them; the row is not linked to article 1, which this scenario does not compare.

Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatingTest.php:140 (Core) (local)
Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatingTest.php:162 (Extbase) (local)
Filetypo3/sysext/extbase/Tests/Functional/Persistence/Facts/Fixtures/Translating/CopiedValues.csv (local)

Prefix lang title prefixes the translated value

Deviates

In 14.3

Core: The English article 1 is localized to German through DataHandler, the backend action "localize". The field label has the language prefix (prefixLangTitle), so DataHandler writes the label of the German article 2 with the prefix "[Translate to German:]" in front of the value of article 1.

Extbase: Extbase has no localize action. The twin creates a new German object (property "_languageUid") with the label of article 1 and persists it with add() and persistAll(). Extbase stores the label exactly as the code assigned it, without the language prefix; the TCA setting prefixLangTitle is ignored.

Target for 15.x

Extbase behaves as the core.

Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatingTest.php:180 (Core) (local)
Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatingTest.php:202 (Extbase) (local)
Filetypo3/sysext/extbase/Tests/Functional/Persistence/Facts/Fixtures/Translating/PrefixedLabel.csv (local)
SkipExtbase differs from core: it stores the label without the language prefix, because it ignores prefixLangTitle.
TestTranslatingTest::prefixLangTitlePrefixesTheTranslatedValueByExtbase (Failure without skip)
Assertion in data-set failed for "tx_extbasefacts_domain_model_article:2":
Fields|Assertion                         |Record
label |[Translate to German:] First label|First label

A synchronized field follows the default record

Deviates

In 14.3

Core: The subtitle of the English article 1 is changed through DataHandler. The field subtitle allows language synchronization and the translation 2 follows its parent, so DataHandler writes the new subtitle into the translation as well and records the state "parent" for the field in l10n_state.

Extbase: Extbase reads article 1 through the repository, changes the subtitle and persists it with update() and persistAll(). Extbase writes the new subtitle into article 1 only; the translation 2 keeps its old subtitle and l10n_state stays NULL, because Extbase does not know language synchronization.

Target for 15.x

Extbase behaves as the core.

Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatingTest.php:270 (Core) (local)
Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatingTest.php:292 (Extbase) (local)
Filetypo3/sysext/extbase/Tests/Functional/Persistence/Facts/Fixtures/Translating/SynchronizedSubtitle.csv (local)
SkipExtbase differs from core: it changes only the default record and does not synchronize the subtitle into the translation.
TestTranslatingTest::aSynchronizedFieldFollowsTheDefaultRecordByExtbase (Failure without skip)
Assertion in data-set failed for "tx_extbasefacts_domain_model_article:2":
Fields    |Assertion            |Record
subtitle  |Changed subtitle     |First subtitle
l10n_state|{"subtitle":"parent"}|NULL

A record for all languages

Same as core

In 14.3

Core: A new article with the language "all languages" (-1) is created through DataHandler. DataHandler writes the row with sys_language_uid -1 and l10n_parent 0.

Extbase: Extbase creates a new object, sets its language to -1 through the property "_languageUid" and persists it with add() and persistAll(). Extbase writes sys_language_uid -1 and l10n_parent 0, like DataHandler.

Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatingTest.php:316 (Core) (local)
Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatingTest.php:335 (Extbase) (local)
Filetypo3/sysext/extbase/Tests/Functional/Persistence/Facts/Fixtures/Translating/AllLanguages.csv (local)

Saving the default tour gives the translation its own copies

Deviates

In 14.3

Core: The title of the English tour 1 is changed through DataHandler and its stops are sent as the backend form sends them: the field stops holds "1,2". The German translation 2 has no copies yet. DataHandler creates the stops 3 and 4 as copies of the stops 1 and 2: they belong to the German tour 2, carry t3_origuid 1 and 2 and the sorting 1 and 2, and DataHandler writes the number of stops, 2, into the field stops of tour 2.

Extbase: Extbase reads tour 1 through the repository, changes the title and persists it with update() and persistAll(). Extbase writes the new title into tour 1, but does not touch the German translation: it creates no copies of the stops, the stops 1 and 2 stay the only stops and the field stops of tour 2 stays 0.

Target for 15.x

Extbase behaves as the core.

Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatedInlineCopiesTest.php:102 (Core) (local)
Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatedInlineCopiesTest.php:120 (Extbase) (local)
Filetypo3/sysext/extbase/Tests/Functional/Persistence/Facts/Fixtures/TranslatedInlineCopies/CopiedForOneTranslation.csv (local)
SkipExtbase differs from core: it creates no copies of the stops for the German translation, so the translation keeps no stops.
TestTranslatedInlineCopiesTest::savingTheDefaultTourGivesTheTranslationItsOwnCopiesByExtbase (Failure without skip)
Assertion in data-set failed for "tx_extbasefacts_domain_model_tour:2":
Fields|Assertion|Record
stops |2        |0

Record "tx_extbasefacts_domain_model_stop:3" not found in database
Record "tx_extbasefacts_domain_model_stop:4" not found in database

Saving the default tour with two translations copies only for the last one

Deviates

In 14.3

Core: The title of the English tour 1 is changed through DataHandler and its stops are sent as the backend form sends them: the field stops holds "1,2". The German translation 2 and the French translation 3 have no copies yet. DataHandler creates only one set of copies, the stops 3 and 4 with t3_origuid 1 and 2, and attaches it to the French tour 3. The German tour 2 gets no copy, although the field stops of tour 2 holds 2, like that of tour 3. This pins a bug in DataHandler, see https://forge.typo3.org/issues/111069.

Core bug With two or more translations, the DataHandler creates the copies of children without language of an excluded inline relation only once and attaches them to the last translation. It finds existing copies by their origin only, without looking at language or parent, so the copies of one translation are taken for those of the next.

Extbase: Extbase reads tour 1 through the repository, changes the title and persists it with update() and persistAll(). Extbase writes the new title into tour 1, but does not touch the translations: it creates no copies of the stops for the German tour 2 or the French tour 3, the stops 1 and 2 stay the only stops and the field stops of both translations stays 0.

Target for 15.x

Saving the default language record gives every translation its own copies of the children, for the DataHandler and for Extbase.

Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatedInlineCopiesTest.php:144 (Core) (local)
Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatedInlineCopiesTest.php:162 (Extbase) (local)
Filetypo3/sysext/extbase/Tests/Functional/Persistence/Facts/Fixtures/TranslatedInlineCopies/CopiedForLastTranslation.csv (local)
SkipExtbase differs from core: it creates no copies of the stops for either translation, while core creates one set for the last translation.
TestTranslatedInlineCopiesTest::savingTheDefaultTourWithTwoTranslationsCopiesOnlyForTheLastOneByExtbase (Failure without skip)
Assertion in data-set failed for "tx_extbasefacts_domain_model_tour:2":
Fields|Assertion|Record
stops |2        |0

Assertion in data-set failed for "tx_extbasefacts_domain_model_tour:3":
Fields|Assertion|Record
stops |2        |0

Record "tx_extbasefacts_domain_model_stop:3" not found in database
Record "tx_extbasefacts_domain_model_stop:4" not found in database
Filetypo3/sysext/core/Classes/DataHandling/Localization/DataMapProcessor.php:1089
$ancestorId = (int)$dependentElement[$fieldNames['origin']];
IssueForge #111069: Copies of monoglot inline children mixed up with several translations
ReviewGerrit 96696, work in progress: failing DataHandler tests, no fix yet

A stop added to the default tour is copied without its origin

Deviates

In 14.3

Core: A new stop is sent as a child of the English tour 1 through DataHandler, as the backend form sends it: the field stops holds "1,NEW1" and the stop is a NEW record with its pid, its title and the uid of the tour in its field tour. The German translation 2 already has the stop 2 as copy of stop 1. DataHandler creates the stop 3 for tour 1 and a copy of it, the stop 4, for tour 2, and writes the number of stops, 2, into both tours. The stop 4 has t3_origuid 0 instead of 3, because DataHandler duplicates the new record in the data map instead of copying it. This pins a bug in DataHandler, see https://forge.typo3.org/issues/111070.

Core bug When a child without language is added to the default language record in the same request, the DataHandler duplicates the new record for the translation instead of copying it, so the copy carries no origin. Later synchronizations cannot find that copy.

Extbase: Extbase attaches a new stop to tour 1 and persists it with update() and persistAll(). Extbase creates the stop 3 for tour 1 with the sorting 0 and writes the number of stops, 2, into tour 1. It does not touch the German translation: tour 2 gets no copy of the new stop, and its field stops stays 1.

Target for 15.x

The copy of a new child in a translation points to its origin, for the DataHandler and for Extbase.

Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatedInlineCopiesTest.php:186 (Core) (local)
Testtypo3/sysext/extbase/Tests/Functional/Persistence/Facts/TranslatedInlineCopiesTest.php:207 (Extbase) (local)
Filetypo3/sysext/extbase/Tests/Functional/Persistence/Facts/Fixtures/TranslatedInlineCopies/CopiedWithoutOrigin.csv (local)
SkipExtbase differs from core: it creates no copy of the new stop for the German translation and writes sorting 0 for the new stop.
TestTranslatedInlineCopiesTest::aStopAddedToTheDefaultTourIsCopiedWithoutItsOriginByExtbase (Failure without skip)
Assertion in data-set failed for "tx_extbasefacts_domain_model_tour:2":
Fields|Assertion|Record
stops |2        |1

Assertion in data-set failed for "tx_extbasefacts_domain_model_stop:3":
Fields |Assertion|Record
sorting|2        |0

Record "tx_extbasefacts_domain_model_stop:4" not found in database
Filetypo3/sysext/core/Classes/DataHandling/Localization/DataMapProcessor.php:559
$populateAncestorIds = array_diff($missingAncestorIds, $createAncestorIds);
Filetypo3/sysext/core/Classes/DataHandling/Localization/DataMapProcessor.php:647
$newLocalizationId = StringUtility::getUniqueId('NEW');
IssueForge #111070: Copy of new monoglot inline child has no origUid in translation
ReviewGerrit 96697, work in progress: a failing DataHandler test, no fix yet