Talk:Import/Catalogue

Considered OK, Derived as new work. No ToS issue

Could someone explain "Derived as new work. " in the phrase "Considered OK, Derived as new work. No ToS issue" used alongside some of the imagery sources used in Haiti mapping. Dmgroom 11:31, 29 August 2010 (BST)

CanVec and GeoBase

Both CanVec and geoBase are okay for ODbL / CT per email from NRCan, http://lists.openstreetmap.org/pipermail/talk-ca/2010-August/003292.html Rw 18:46, 31 August 2010 (BST)

Geogratis

GeoGratis data is okay per email from NRCan, http://lists.openstreetmap.org/pipermail/talk-ca/2010-August/003235.html Rw 18:49, 31 August 2010 (BST)

SPOT Pakistan

Currently SPOT Pakistan images are listed as "OK" under the contributor terms. Can someone explain how the sentence "to integrate the DERIVATIVE WORKS in the OpenStreetMap database under CC-BY-SA licence (http://creativecommons.org/licenses/by-sa/2.0/) and/or Open Database licence (ODbL: http://www.opendatacommons.org/licenses/odbl/OpenDBL)" meets the conditions of Clause 3 of the CT? I.e. how does the express licensing of CC-BY-SA and ODbL allow for relicensing under any "free and open license"? Also "granted by Spot Image a limited, non-exclusive, non transferable, licence" feels incompatible with Clause 2 of the CT, although with the license being on the images rather than the derivative work, that part might not be as problematic.

- User:Amm 12:09, 29 September 2010‎

Surrey Air Services

Surrey Air Services is currently listed as ODbL/DbCL only. This seems to preclude it from being OK with the CT. Is there a more broad license offered than just ODbL/DbCL?

That thought had crossed my mind too. The data was released under ODbL as it was believed this would be helpful to OSM, but technically that makes it incompatible with the CTs. I consider there is a loop hole in the licenses when ODbL is applied to aerial imagery tiles which were supplied: if the "database" is the file structure and the "contents" are the images themselves covered by DbCL, we can pretty much use the images as we please. However, we should verify the owners of the data are happy with that. --TimSC 20:12, 25 March 2011 (UTC)

Cleanup Request

We should transform the tables to a template approach to make it more understandable --!i! 12:55, 16 November 2010 (UTC)

Better to seperate from "Import done" and "Import in Progress" --Deltabrasil 15:40, 24 December 2010 (UTC)

I propose splitting each row in all tables using subpages, ench one containing a call to a template containing a ... statement to return a specific type of information. With those templates, it will be possible to categorize each import source, add new type of information if needed for the future, display each source in a summary presentation linked to a full presentation page.

My idea would use the following type of code :

<includeonly>{{#switch: {{{info|}}
| name = ...
| description = ...
<!-- and so on... for other information types -->
| #default = /?/[[Category:Import/Catalogue entry missing {{{info|}}} information]]
}}</includeonly><noinclude>
{{Import/Catalogue/entry}}
</noinclude>

The noinclude section calls a common utility template that will generate a detailed and readable view of the information exposed, it will test for the presence or absence of mandatory information fields, it will autocategorize the subpage containing this code into useful categories (according to the licencing status or approval, or conditions for using them).

As these subpages will be working effectively like templates (but not generic enough to be used as regular templates for every kinf of pages), it will be possible to generate with them several report pages, by just enumerating their subpages names (in the Import/Catalogue) using a report-specific presentation template which will present some fields, for example in a summary table (with a link to the complete subpage showing all details).

With specific report pages, it will be possible to add further information field types, validating their given value

For subpages where an information field is not specifying an explicit value, these will return a default value string starting by /?/ which is easily testable using the Mediawiki #titleparts ParserFunction as : {{#ifeq:{{titleparts:''value''|2|1}}|/?/|...}}).

As the missing info fields are subcategorizing each missing info field in a separate category, it is possible to create also a report page in this category, explaining what is the expected value and its usage policy. That report page may perform other validations on the list of subpages present in that category.

Note that a MediaWiki extension (Extension:PageAfterAndBefore) allows enumerating all pages in a specified category, using a processing loop (Extension:Loops), allowing for example to call a specified template for each page listed in a specified category, passing it a parameter for the name of a page found in that category, so the maintenance of lists of subpages may be simplified a lot.

1. If those two MediaWiki extensions are not installed, you'll need to provide in each report page a list of the subpage names to report, and maintain those lists in every catalogue report page (such maintenance may be automated by using a bot). If there's only a single report page (the Import/Catalogue itself), it is acceptable and easy to edit it in a relevant section, to insert the catalogue entry, without much maintenance and without needing any bot (this method what is typically used, for example in most wikis for page deletion requests which use a subpage for each request).
2. If those two MediaWiki extensions are used, the loop may become very long to process if the category containing all Import/Catalogue subpages becomes highly populated: in this case the import catalogue should be subdivided in separate subcategories. For this reason, the maximum number of loops should be limited (this should not be a problem is the Template:Import/Catalogue/entry (as used above) correctly performs some basic autocategorization by validating a few information fields using conditional #switch: or #if: tests, so that each subcategory will contain less than 200 catalogue entries).

In an initial version of the cleaned up page of the Import/Catalogue, I would simply use option #1 above, without using these MediaWiki extensions, even if there are a few other reports maintained (in the same page or in another report page).

As these will be separate subpages, each one will have its own talk page (containing also some decisions made by the community) about the status and conditions of use. And each subpage will also be categorizable manually (if this is not performed automatically by the call to {{Import/Catalogue/entry}} present in the noinclude section above for showing the full details specified in that catalog entry.

— Verdy_p ^(talk) 20:22, 7 November 2012 (UTC)

This has been discussed for 2 years. Has anything been done? I support Verdy_p's proposal and suggest we step forward and do it. If it doesn't work, we can roll it back. Jeffmeyer 01:34, 7 December 2012 (UTC)

Pending Verdy_p's update, I'm trying to see if there's an easier way to do display this table on a typical wiki page: Import Catalog Draft Jeffmeyer 02:37, 21 December 2012 (UTC)

Please see this Google spreadsheet, which contains an attempt to clean up & standardize this data: https://docs.google.com/spreadsheet/ccc?key=0As7xcVwkvHhtdHRIUF9sSzlOdVQ5ZWZPZnlJUlU1S3c

General thoughts on this page right now:

* It's way too crowded & tries to fit too much information across the page

* It's probably too long for an individual page

* If we were to have a list, I'd suggest including only the following information: Name, Country, Contact, whether the import has been reviewed by others, and status of the import

* Other information that should be included on the import wiki page: Link to data Link to data license, Attempt to characterize the type of license (if not explicit), Have someone verify compliance with ODbL, Contributor Terms - etc. - not sure I understand what that means, Link to Contributors page statement when attribution is required, Copy of any correspondence related to permissions, etc. Jeffmeyer 18:08, 21 December 2012 (UTC)

Why do we have a column for compatibility with Contributor Terms?

Contributor Terms are an agreement between users of OSM and OSM. Why is this being applied to data? If there's something specific that ties the two together, we should add a description identifying why this additional compliance is required beyond the data having an appropriate license. Jeffmeyer 01:36, 7 December 2012 (UTC)

I also agree that this is not necessary. Users (or their bots) doing imports must necessarily agree with contributor terms to perform their uploads. What then makes these imports legal is the copyright/licence info and or explicit agreement where needed to import it with another licence compatible with OSM rules.

This column unnecessarily clutters things. We already have a column for users responsible of these uploads and that asserted (according to their own acceptation of Contributor Terms) to respect the copyrights and licences and the community decision about their import process and documentation.

I note however that some imports are missing info un the User column, but the link in the first column normally goes in that case to a page on this wiki describing the dataset, or there are other links in the page providing copies of agreements (emails, forum messages, archived media file for scanned letters...)

These tables are already very large. For some cases, very long URLs have been replaced by "[URL title]" so that their "title" part may wrap where needed (sometimes I had to insert a few forced linebreaks)

It is still possible to compact it a bit more, and dates could be uniformized (preferably using ISO format so they become sortable?)

Note that some users have confused their Wiki user name and their OSM user name (so you still see red links for them, notably for bot accounts used for data imports, which are generally not wiki user names: I did not change them, they are still red). I used the {{OsmUser}} template instead of plain URLs (for HTTPS compatibility).

I have also realigned some columns which did not match together. There are no longer any empty cell (empty cells or cells previously with "-" now display "N/A").

Probably items in these lists should use a template with explicitly named parameters, including one with a static shared parametric value for the presentation wanted, and this page would show one format, with other pages inclusing the same lists with with another format value (the format would be specified in the subbpage part of the template name with a default ({{Import/Catalogue/Entry/{{{format|row}}} | name=... |status=... |user=... |start=... |end=... |licence=... }})

Finally the column for areas should be starting by the Country name, followed by subregions and city name, so they are sortable more or less geographically and become more easily searchable. — Verdy_p ^(talk) 00:03, 17 April 2017 (UTC)

The ODbL OK Status column could probably go to since its also kind of a given the person doing the import has to agree to it for their data to added. So there is really no need for it since its implied. Maybe a note in like "in order to do an import users must agree to both the ODbL license and contributor terms" or something similar could be put in the introduction paragraph instead.

Per the discussion here, I have removed these columns. Nate Wessel (talk) 05:56, 14 October 2019 (UTC)

It would also help if there was a short paragraph somewhere explaining standard formats for the cell data, like ISO for the dates as suggested or shorting urls. That way things would at least be more uniform and probably sort-able. Plus the amount of characters could be cut down that way too. Adamant1 (talk) 10:55, 8 June 2018 (UTC)

There is a table waaaay at the bottom that explains this. It should probably be made more prominent though. Nate Wessel (talk) 05:56, 14 October 2019 (UTC)

Should we link to the appropriate contributor license statements?

For example, if we have permission to use this data, we should have a quick link to proof of that permission, no? For example, http://wiki.openstreetmap.org/wiki/Contributors#Toronto Jeffmeyer 01:39, 7 December 2012 (UTC)

Why are images included on the import page?

It's not clear why digital orthophotos are included here - is map tracing considered a form of imports? If so, shouldn't Bing be listed on this page? Jeffmeyer 08:13, 27 December 2012 (UTC)

No. I agree. These are not imports unless we widen the definition in an unhelpful way. Given how long and unwieldy the page is, that section should be moved off to somewhere better. That leapt out to me as obvious cleanup to do here.

I will move the digital orthophotos section off here and onto Talk:Aerial imagery#List moved from Import/Catalogue for the moment. Then maybe someone can pick through the info and check if there's anything missing from the lists on the Aerial imagery page (That mostly seems like the better place for this I think)

-- Harry Wood (talk) 23:41, 19 March 2017 (UTC)

DONE

but there's also a section "'Digital Rectified Maps" which is the same problem. Raster map layers. Not imports.

Since these are all well described on individual wiki pages NPE, 7th_Series, Provisional/First_Edition, Provisional/First_Edition. I think just blowing away this table on this page will not lose much of any value

DONE that too.

-- Harry Wood (talk) 00:08, 20 March 2017 (UTC)

Order of items in Area column should be reversed

Why would it make sense to have values like these in the Area column?

• Montagna in Valtellina (Sondrio) - Italy
• West Virginia, USA
• Various Cities and Counties in USA
• Fortaleza, Brazil
• Montréal, Québec

These values could, instead, be:

• Italy, Valtellina (Sondrio), Montagna
• USA, West Virginia
• USA, Various Cities and Counties
• Brazil, Fortaleza
• Canada, Québec, Montréal

If the Area column value did not go from most specific name to least specific name, but instead went the other eay, one could sort and see, for example, all the US or Brazil-specific imports. It would be much more helpful.

I could do the edits, of course, but I would want to hear that the community would not object.

--RayKiddy (talk) 17:39, 11 October 2021 (UTC)

@RayKiddy: Sounds like great idea, but be aware that updating those may take long time. This spring I was only updating date formatting and even then it took some hours. Considering this page has had fixme/cleanup template for probably long time, looks like most of clean-up activities haven't even sought approval as long as information itself wasn't moved away from the page. If you have time, perhaps add region/continent information to Area column and maybe also look into why some tables' rows have varying number of columns or if some of those "ongoing" imports from 2012 are still going on. --Fghj753 (talk) 14:25, 16 October 2021 (UTC)

@RayKiddy: I suggest drilling down into the subcategories of Category:Import by country instead. – Minh Nguyễn ^💬 03:39, 25 May 2023 (UTC)

Contributor Terms column has been removed

While cleaning up the page the Contributor Term (CT compat.) table column was so mangled and messy that it was just better to remove it. The column also appeared to be redundant as most if not all of the useable data it provided was just a duplicate of the ODbL. If it is ever added back it would have to be done properly and most importantly be useful. Mxdanger (talk) 08:05, 15 March 2023 (UTC)

@Mxdanger: FYI, this column was the result of a vote of the OSMF board. [1] I'm not sure what the board had in mind for maintaining the column beyond the act of adding it back in November. There used to be one several years ago, but it was also removed as unmaintainable and redundant. – Minh Nguyễn ^💬 03:38, 25 May 2023 (UTC)

Breaking up this catalog

Over on the community forum, I've proposed deprecating this wiki page in favor of the existing imports category. I suspect the pages in that category are already more comprehensive than this catalog and would be easier to maintain in the long run. – Minh Nguyễn ^💬 17:59, 21 May 2023 (UTC)

Difference between "Community imports" and "One-Time Imports"

Hi, I'm struggling to understand the difference between these two categories. Could someone explain it to me? Thanks --Ivanbranco (talk) 09:59, 22 May 2023 (UTC)