tag all
caps by treating it as startTagR14.
- Replaced throw new InternalError by AssertionError to fix bug
#35 Internal Error
- Ignoring virtual xml tags in WtPrettyPrinter (Caused problems in issue #44)
- Differentiating between italic/bold started by html tag or ticks thus partly
fixing issue #44
## 3.1.2 - 2017-01-16
### Changed
- Bumped version of tooling parent pom and osr-common dependencies to 3.0.4
## 3.1.1 - 2016-08-12
### Changed
- Made ScopeStack class and methods public (including inner class Scope)
## 3.1.0 - 2016-06-13
### Changed
- Document automatically generated during deserialization in
sweble-wom3-json-tools if not doc is explicitly set does not contain article
element by default any more.
- Generalized sweble-wom3-json-tools code to work with w3c docs as well (BREAKS INTERFACE)
- Generalized sweble-engine-serialization code to work with w3c docs as well (BREAKS INTERFACE)
- Generalized Wom3Toolbox code to work with w3c docs as well (BREAKS INTERFACE)
### Added
- WomToolbox.{isWomElement, isRtd, isText, isRtdOrText} methods
- WomSerializer.setDocumentImplClassName method
## 3.0.2 - 2016-06-07
### Changed
- Bumped version of tooling parent pom and osr-common dependencies to 3.0.3
## 3.0.1 - 2016-05-03
### Fixed
- Illegal characters had no rtd assigned
### Added
- Added parser configuration options to turn off foster parenting in
post-processing
### Changed
- Bumped version of tooling parent pom and osr-common dependencies to 3.0.2
[keepachangelog]: http://keepachangelog.com/
sweble-wikitext-sweble-wikitext-3.1.9/etc/ 0000775 0000000 0000000 00000000000 13352705604 0020560 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/etc/agpl-v3-header-no-copyright.txt 0000664 0000000 0000000 00000001167 13352705604 0026445 0 ustar 00root root 0000000 0000000 This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see http://www.gnu.org/licenses/.
sweble-wikitext-sweble-wikitext-3.1.9/etc/agpl-v3-header.txt 0000664 0000000 0000000 00000001327 13352705604 0024023 0 ustar 00root root 0000000 0000000 Copyright 2011 The Open Source Research Group,
University of Erlangen-Nürnberg
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see http://www.gnu.org/licenses/.
sweble-wikitext-sweble-wikitext-3.1.9/etc/apache-license-v2-header-no-copyright.txt 0000664 0000000 0000000 00000001013 13352705604 0030350 0 ustar 00root root 0000000 0000000 Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. sweble-wikitext-sweble-wikitext-3.1.9/etc/apache-license-v2-header.txt 0000664 0000000 0000000 00000001153 13352705604 0025735 0 ustar 00root root 0000000 0000000 Copyright 2011 The Open Source Research Group,
University of Erlangen-Nürnberg
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. sweble-wikitext-sweble-wikitext-3.1.9/etc/checkstyle.xml 0000664 0000000 0000000 00000024474 13352705604 0023453 0 ustar 00root root 0000000 0000000
sweble-wikitext-sweble-wikitext-3.1.9/etc/forbiddenapis.txt 0000664 0000000 0000000 00000000071 13352705604 0024130 0 ustar 00root root 0000000 0000000
# Deprecated. Use org.junit instead.
junit.framework.**
sweble-wikitext-sweble-wikitext-3.1.9/icon/ 0000775 0000000 0000000 00000000000 13352705604 0020735 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/icon/sweble.png 0000664 0000000 0000000 00000030707 13352705604 0022733 0 ustar 00root root 0000000 0000000 PNG
IHDR Xۘ sBIT|d pHYs ' '\! tEXtSoftware www.inkscape.org< IDATx{x]Wynƶlcb] HB!ĒC 0P.R0z ah)RLt&!Kv1 $.N:"Y ->HHsJy%'iإ۽ _?:
o\ߒY2sؓDVK-*umh\4L1gQ{{ʩf9<疉ܸ7e4LDmǛ\W<Ӳb1ߢĆ{تp5ܲ'Z<{P|̉IOwSF~{LBY>J2}圻sY=32?fQt 2ЎznO}XSf钉Ƨd8/<:%{Xl[,I cE ZAdc㉃ 3i_̆η0~c֠#9 ' 64GGN=gGܢN}kR9ӟϲHAEߩG'`ֳ Hv.7
p&?2`&czv
EQy@A ~ODnHջUhtϖ٥:l1IL7<
ny{"cj\<<5ca.Jj5U=i͒r
.IysA9A4u) /*TCKtaC3 Wo+96 H7wzRݩe5g]H$K7a6XM̚0zS+,q&5¾\Sё oľ-r e_`-rFoe;ZI64ސ|BNmJ%^>a 4&d /ŋm;@O^!Y2(^2~ lL'$.DǕֻ:Y7d0~ ᱫR2'?i';O5A }YG zPf'{p}_ Ols35亭sGFeý#U\ɉ̉^R .Pov)r
;?E vw0U8ܾ r7yk!k4Vy2m4cfp~~egT{۞ ^xwOw:;.s;7oó19ГzI[\oؑ$4S 1cq d&[1.qj2T_-$s'zI_)qS0|yw {+V_4$ ^WgGH+Rߖ8]rWԡ %"ѭ֨2j\/En{by5 a62:eo j9r:!Y&W\2 mV 4B6C$(PG8qixI*.lvwgHѯSLͶJG(iY˒$ Y^弅>ZMc)I _??PCKEנ{_\2
Ŝ4a߬f{AuŬ脰'Q:$KI:C qKWXiEXJRG-IҲfzZʿd khG7cc)I@?puԹ3}gcN/tSm9I?3R9`' KekbOfM, ӏ+fhמ;xD
i>I?S6cw
2ڝ: niiK?~3QܕSq
G醾OK^-ⵈfܿL2Wnpy%Ymgݝ$
7'F>w̡ISF%:v .B=^%/ q!.z[3en5 o\"^.׳QH5
_&i{f6#Rl 'QCա]{#^/Ҥ)ݽ]Ɵ^9 [Vbrߩ#"$\LIt1Q+{tb'eV(O'4V۾}& %{9/UihX
e,Z gE"GU828ۦ(&g{r,Wim;"FѮ :$Ἳ
<4^}JIeMȉ.Y.}eVe/uXv]+yS :沛g.sճؗ6P$ݽJ
'N%h3;`.k G=pI/3kRM?pΎj:oߤq4\}\ 6diibe%mM'^[mKIڛ琅տ w E/ݽ¤냹$w=Jw!3*|*OB % a찬>,km`n}a)Cnё Ϧzd$EMו[uA\tU0uH+;}[%+Ӥ(,xt|VeH)I֘E/{Nbe}j>1i: .I3#Yu
(` 0|/D#/U7San]e%B]g&MpY]PQܰgИQòX
yYS* }ǾK3+a.
A[)Xh%^IʤE+j|aCȏ%
Ox@B6Z-^XmkK>'˺Z/ӼȽ]'n̬S{]f'y!PYdc Tt{MsJY7$u⎯QT% R6n3ge s*YB#,`j %}'6e30_kU$"/ElO=@hc?2k 1)s\D =7kw%yΝSK*6%1Q]7$7ԳI\U(>;Vmq0< }ͯL?ٟ~Cv!~+U]^ִ wIQPξCr*foƥZ%$ڙ~@3'$ Zr<\z3HEGo.m[iZ{!1zס^zblf߯Au0OtZ^ T+*# ns<=[6':ѺbeK͎U2~y,V#I8)&+\5ihӪ_tv,
{>4KG)@Q[~eM%-K nJzsjN+gq˦n9Cuf涗G-GbR ( &,PatfN?Spvcn?4k5c{|;ݽ6!+&$PY%v͢[yc%)j9I3RLFA`2L[}#%^V\v8
1 _Ӻ>u]~
~fOw*W%) p@X1FRhg:)|[gL2sF
{Zyy6G[\Yۯ nURg[% }OMe4i?+O.aաJOKyRg_=)_Lo ibC>u |ݺLoʽWMBJkf_ ~RY_ڲKTܺIRk g M_IsCGԡ" fƫ0fowowEP>wc7DZl
\gf
$u?6۶WW @5ݙPoI}*7s-VD7R k~s7-u9 o7pb'u[ɥ*R4.@9L,=H9iK?Г+0|@c:yeiCdFOZۚ'
;wuSHE\T{ beMò-~Nφqʅ
}OHR&֘jd+`L ǖ゛iNʤ7m}=?6{̽ U:];*{g*7{zZ?+~]KY
m7i Vdg?5u__R)٦0Er;cE %eIK3U?q#% Ty=nVHefГ^?Л\Rab.{O. شڮc˃sKa/URhߖu2 Bo`oN~ڹ+9;6fLhY[42/z #e rYQC@~k${*oo^?ԝ /j]> SlW_otșdoTzv z͕ݵ6{VJZUi٩]<*j0dz͚ I/ .d7e]ZW[y
wL(F]_t/,}\,GeiQHZ>]S8ܳv}d UYnPO懚Mghs飝}WRxV۪_Ќ$Y<|:G߬@H^˽\n#W=N39!~Xvj=eټh:s8,nX6^qXh8t#c ;Fj *3/aM Jw*
}ҷ
O㦡s$(>;tlO? ؘ+͜E`?zGڨxy
MѢ{)V;r^:J$+۾c1 IϬG]暎 #U x,;>yg>QPbN7']27
(,uo|Tj*E_5އ^*>ދle*=) _֧xa0s%7.5 ±~93sLirbT$e^2zEUl1x.\Y' IoRuZ&MIթ7miXٯwdV(++=i*,=N墢KYJcC# ;E??ܳ?KR5; fO'Mۙƃ=&{ /N[ E&52
Ζ=wѪWj2=r:9Toܲbo ^[6 |*9kךMy_5x|s17l1\npkLZɮ1_̛\m7.}{r|IWZ/v;r3>ؓK+/mشfI =(
ƗmZ QU'jIyx!9Se0 f gLY
/Q̈
k*S۬ek+f0Ojߵg<S k~qu/vXv
]*#Z
w7%bò±"±kȀ'%Zm)My>t}Y/if/ y
^&, V7m3T
v / gI$]yu)IAsFiL%_ݱ\ӳE2= 8IDATWޛ]Чx |<ѓ4,aU±G'V9 Wgy`vÐ'E$b]5 |pt²/OZwǾ1t%[乐/QŝwPdX})IgÆkilKqòמ Tҥ w`0-e lfBicQ %cSy,,Uϒ}p&0=xr'OlݻSݢW[[TSU96w/ս'TQ8Ң ^4
wH_;i'v?pS`A{Kk%-*p / Fg_^UES\*JI Y~%pV_%stJIOb&P(?-n1Y[]$I{9ewb-sSI ؎8e]Hn[yq⍳>cfǚf±rr%=Ј~\ԍ٦ IUV`31)^>m$VX{&{6yY
:+0nn>zWw@J2T4d3_M 1t~4 XkxOUukxӝo]^٬WBqdue&N|be]oT/I+vPЇK_\x;nYw$2WY;5 w"í3<U u,SkI7T{Q:LKx*SsnO 1)VX&T1;FKT . +Ir{[ܢfV5 }%ߘH`ÔL7zvI 3]?/Pl^W =ig}I]O ߪPf{0flxs%2f&
u|':5-G[Wҹ3WPS'/s7M )phu؇~9(@gJD}Mò=aD _vbӆ~[
09ޣ̱/T1<4ԭ3fD.Ds;7+tn_v7fڻErm+peZP=k%] `l瓭c1i]ch06\^$0' -p)],Z{T0̓_ IsK&,k$>t <((n6I]Sj.CW(Bzw=IEtڦ i}[q;)Pݽ]Q?ВH UpI>Do}Vu)W]DyV)38ؤ
uL/|Fo^*+Va[cUIzm/n߸iw?W~f*eysLi*e m{w"5դul X=\ývDՅeM_oyMqs+&{W]淩Xh#U5|9ZU1{{N,O+:)fNIp{w3Iz✛ ~7+KT>=αbJ"{9IC-&{ͣ*d pIIe}^c8|vzINZ>e^]ouVbU)m{w]W,L-Y\ UC#T'hqYBT.l7rYsrDzى4پ2N /> :
t*>-IfV(sbeM7R Ue{Fޓ%iu'۲n@wE3KRmU
zۮ /cs*9dE=55onJ
I}\-d=oֳFJY~'uJc&=gSqQ=M{IRdgN6{KˑeC@G_Jo˺cHdNxo;|x'
Qj;2$ywk(ژ\ 5.k 7XffVv#fa.@oo1첡#[u=RV zUjY {ɻ%_ޒ#2{ogI*.S+im=ܓk
J odȃvIKzd\'f%IG߯NN!Ld 0m5v]+s;HJl 734*lsuPOj_ܝ ^նCn,3 Imn-Ŷ*,Wݱ`okܮUw۲nL浊r7I#w )%_txo/j'mQ[{R 3[.-i%ujB]2v}$+ՊӚb{`K0W$
Lnkq
lZuYr^o &V#r4lnԂʣ.ml7L -Ork=Cn/|m?zWǎbZ2_O7ITa?{c}¡#m,mZ$=s,3qޖl,t;5^
kXfy
+_ 邧|7yֿFfLbxbwoW,M|O.X=
E^նAa!%yNh&KdPyb"GMyS(ify^}-H_|rN%>7[?\?WfEP:ŕ( |VVOP|}Go)]G-n6E%zίʹYR"%`NyRR䏺ydY3d~ PN&?L
Q3]; i.V2헎}m| {ӜsuW<9ʤz!G$/i#/0冾|֠xi Ky\'\$nUuP/7-]X/pȭK 2uHZ<[qDO%vϽC j{usʟvC|!WP{mnor*2ܳv}&\_sCW^m09-qoʌMDanݻ3rɂtL =+h. 0 ?&ʽ IENDB` sweble-wikitext-sweble-wikitext-3.1.9/icon/sweble.svg 0000664 0000000 0000000 00000021051 13352705604 0022736 0 ustar 00root root 0000000 0000000
sweble-wikitext-sweble-wikitext-3.1.9/pom.xml 0000664 0000000 0000000 00000023746 13352705604 0021336 0 ustar 00root root 0000000 0000000
4.0.0
de.fau.cs.osr
tooling
3.0.8
org.sweble
sweble-parent
3.1.9
pom
Sweble Parent
2010
The next generation Wiki platform project.
sweble-engine-parent
sweble-wikitext-components-parent
sweble-wom3-parent
${project.basedir}
${sweble-parent.path}/etc
${sweble-parent.etc-path}
true
3.0.9-SNAPSHOT
3.1.9
de.fau.cs.osr.utils
utils
3.0.8
de.fau.cs.osr.utils
utils-testing
3.0.8
de.fau.cs.osr.utils
utils-getopt
3.0.8
de.fau.cs.osr.ptk
ptk-common
3.0.8
de.fau.cs.osr.ptk
ptk-xpath
3.0.8
de.fau.cs.osr.ptk
ptk-xml-tools
3.0.8
de.fau.cs.osr.ptk
ptk-json-tools
3.0.8
org.sweble.wikitext
swc-parser-lazy
${sweble.version}
org.sweble.wikitext
swc-engine
${sweble.version}
org.sweble.wikitext
swc-dumpreader
${sweble.version}
org.sweble.wikitext
swc-article-cruncher
${sweble.version}
org.sweble.wom3
sweble-wom3-core
${sweble.version}
org.sweble.wom3
sweble-wom3-json-tools
${sweble.version}
org.sweble.wom3
sweble-wom3-swc-adapter
${sweble.version}
org.sweble.engine
sweble-engine-serialization
${sweble.version}
com.mycila
license-maven-plugin
${sweble-parent.etc-path}/apache-license-v2-header.txt
${sweble-parent.etc-path}/apache-license-v2-header-no-copyright.txt
etc/apache-license-v2-header-no-copyright.txt
etc/agpl-v3-header.txt
etc/agpl-v3-header-no-copyright.txt
sweble*/**
The Apache Software License, Version 2.0
http://www.apache.org/licenses/LICENSE-2.0.txt
repo
GitHub
https://github.com/sweble/sweble-parent/issues
scm:git://git@github.com/sweble/sweble-parent.git
scm:git://git@mojo-forge.cs.fau.de/sweble-public/sweble-parent.git
https://github.com/sweble/sweble-parent
sweble-wikitext-3.1.9
true
true
daily
warn
osr-public-repo
OSR Public Repository
http://mojo-maven.cs.fau.de/content/repositories/public
jenkins
env.BUILD_NUMBER
org.codehaus.mojo
cobertura-maven-plugin
${cobertura-maven-plugin.version}
xml
true
true
package
cobertura
sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/ 0000775 0000000 0000000 00000000000 13352705604 0024020 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/LICENSE 0000664 0000000 0000000 00000026136 13352705604 0025035 0 ustar 00root root 0000000 0000000
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/pom.xml 0000664 0000000 0000000 00000006167 13352705604 0025347 0 ustar 00root root 0000000 0000000
4.0.0
org.sweble
sweble-parent
3.1.9
org.sweble.engine
sweble-engine-parent
pom
Sweble Engine Parent
2013
The Sweble Engine modules provide components required to build a wiki
engine.
sweble-engine-serialization
sweble-engine-serialization-eval
${project.basedir}
${sweble-engine-parent.path}/../
com.mycila
license-maven-plugin
${sweble-parent.etc-path}/agpl-v3-header.txt
${sweble-parent.etc-path}/agpl-v3-header-no-copyright.txt
GNU Affero General Public License
http://www.gnu.org/licenses/agpl-3.0.txt
repo
sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/ 0000775 0000000 0000000 00000000000 13352705604 0032344 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/pom.xml 0000664 0000000 0000000 00000014047 13352705604 0033667 0 ustar 00root root 0000000 0000000
4.0.0
org.sweble.engine
sweble-engine-parent
3.1.9
sweble-engine-serialization-eval
jar
Sweble Engine Serialization Evaluation
2013
A module for evaluating the serialization module's performance.
../
true
true
true
true
org.apache.commons
commons-math3
commons-io
commons-io
xerces
xercesImpl
net.sf.saxon
Saxon-HE
com.google.code.gson
gson
de.fau.cs.osr.utils
utils
org.sweble.wikitext
swc-parser-lazy
org.sweble.wikitext
swc-engine
org.sweble.wom3
sweble-wom3-core
org.sweble.wom3
sweble-wom3-swc-adapter
org.sweble.engine
sweble-engine-serialization
de.fau.cs.osr.utils
utils-testing
junit
junit
com.mycila
license-maven-plugin
src/test/resources/**/*.wikitext
org.apache.maven.plugins
maven-dependency-plugin
analyze-dependency-usage
net.sf.saxon:Saxon-HE
xml-apis:xml-apis
de.thetaphi
forbiddenapis
commons-io-unsafe-1.4
sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/ 0000775 0000000 0000000 00000000000 13352705604 0033133 5 ustar 00root root 0000000 0000000 main/ 0000775 0000000 0000000 00000000000 13352705604 0034000 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src java/ 0000775 0000000 0000000 00000000000 13352705604 0034721 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/main org/ 0000775 0000000 0000000 00000000000 13352705604 0035510 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/main/java sweble/ 0000775 0000000 0000000 00000000000 13352705604 0036771 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/main/java/org engine/ 0000775 0000000 0000000 00000000000 13352705604 0040236 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/main/java/org/sweble serialization/ 0000775 0000000 0000000 00000000000 13352705604 0043113 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/main/java/org/sweble/engine SerializationTestBase.java 0000664 0000000 0000000 00000004365 13352705604 0050236 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/main/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import org.sweble.wikitext.engine.EngineException;
import org.sweble.wikitext.parser.parser.LinkTargetException;
import org.sweble.wom3.swcadapter.utils.WtWom3Toolbox;
import org.sweble.wom3.swcadapter.utils.WtWom3Toolbox.Artifacts;
import de.fau.cs.osr.utils.TestResourcesFixture;
import de.fau.cs.osr.utils.WrappedException;
public abstract class SerializationTestBase
extends
SerializationLabToolbox
{
private final TestResourcesFixture resources;
private WtWom3Toolbox wtWomToolbox = new WtWom3Toolbox();
// =========================================================================
protected static TestResourcesFixture getTestResourcesFixture()
{
try
{
File path = TestResourcesFixture.resourceNameToFile(
SerializationTestBase.class, "/");
return new TestResourcesFixture(path);
}
catch (FileNotFoundException e)
{
throw new WrappedException(e);
}
}
// =========================================================================
protected SerializationTestBase(TestResourcesFixture resources)
{
this.resources = resources;
}
// =========================================================================
protected TestResourcesFixture getResources()
{
return resources;
}
public Artifacts wmToWom(File wmFile) throws LinkTargetException, IOException, EngineException
{
return wtWomToolbox.wmToWom(wmFile, "UTF8");
}
}
TimeSerialization.java 0000664 0000000 0000000 00000030644 13352705604 0047421 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/main/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
import java.io.File;
import java.io.IOException;
import java.nio.charset.Charset;
import java.util.List;
import org.apache.commons.io.FileUtils;
import org.apache.commons.math3.stat.descriptive.moment.Mean;
import org.apache.commons.math3.stat.descriptive.moment.StandardDeviation;
import org.apache.commons.math3.stat.descriptive.rank.Median;
import org.sweble.engine.serialization.CompressorFactory.CompressionFormat;
import org.sweble.engine.serialization.WomSerializer.SerializationFormat;
import org.sweble.wikitext.engine.EngineException;
import org.sweble.wikitext.parser.parser.LinkTargetException;
import org.sweble.wom3.Wom3Document;
import org.sweble.wom3.swcadapter.utils.WtWom3Toolbox.Artifacts;
import de.fau.cs.osr.utils.SimpleConsoleOutput;
import de.fau.cs.osr.utils.StringTools;
import de.fau.cs.osr.utils.TestResourcesFixture;
import de.fau.cs.osr.utils.WrappedException;
public class TimeSerialization
extends
SerializationTestBase
{
private static final int WARM_UP = 100;
private static final int ITERATIONS = 100;
private static final String INPUT_SUB_DIR = "/various";
private static final String FILTER_RX = ".*?\\.wikitext";
private static final CompressionFormat[] compressionFormats = {
CompressionFormat.BZIP2,
CompressionFormat.GZIP,
CompressionFormat.XZ
};
private static final SerializationFormat[] serializationFormats = {
SerializationFormat.JAVA,
SerializationFormat.JSON,
SerializationFormat.XML
};
private static final Charset CHARSET = Charset.forName("UTF8");
// =========================================================================
public static void main(String[] args) throws IOException, CompressionException, SerializationException, DeserializationException, LinkTargetException, EngineException
{
new TimeSerialization(SerializationTestBase.getTestResourcesFixture()).run();
}
// =========================================================================
private final WomSerializer serializer = new WomSerializer();
public TimeSerialization(TestResourcesFixture resources)
{
super(resources);
}
private void run() throws IOException, CompressionException, SerializationException, DeserializationException, LinkTargetException, EngineException
{
TestResourcesFixture resources = getTestResourcesFixture();
List inputs = resources.gather(INPUT_SUB_DIR, FILTER_RX, false);
for (File file : inputs)
{
String title = file.getName();
SimpleConsoleOutput.printBigSep(title);
Artifacts af = timeParsing(file);
Wom3Document wom = af.womDoc;
System.out.println();
long uncompressedWikiMarkupSize = getSize(af.wm);
printSize(2, "wiki markup", uncompressedWikiMarkupSize);
for (CompressionFormat compressionFormat : compressionFormats)
{
System.out.println();
System.out.println(" Compression format: " + compressionFormat);
System.out.println();
long compressedWikiMarkupSize = getSize(compress(af.wm, compressionFormat));
printSize(4,
"compressed wiki markup", compressedWikiMarkupSize,
"uncompressed wiki markup", uncompressedWikiMarkupSize);
for (SerializationFormat serializationFormat : serializationFormats)
{
System.out.println();
SimpleConsoleOutput.printSep(4, "Serialization format: " + serializationFormat);
for (int format = 0; format < 4; ++format)
{
boolean compact = ((format & 1) == 1);
boolean pretty = ((format & 2) == 2);
if (!compact || pretty)
continue;
System.out.println();
System.out.println(" "
+ "Compact: " + compact + "; "
+ "Pretty: " + pretty);
System.out.println();
byte[] serialized = timeSerialization(wom, serializationFormat, compact, pretty);
timeDeserialization(serialized, serializationFormat, compact, pretty);
if (!serializationFormat.equals("JAVA"))
saveSerialized(title, serializationFormat, compact, pretty, serialized);
System.out.println();
printSize(8,
"serialized WOM document", getSize(serialized),
"uncompressed wiki markup", uncompressedWikiMarkupSize);
System.out.println();
byte[] compressed = timeCompression(serialized, compressionFormat);
timeDecompression(compressed, compressionFormat);
System.out.println();
printSize(8,
"compressed WOM document", getSize(compressed),
"compressed wiki markup", compressedWikiMarkupSize);
System.out.println();
compressed = timeCompressedSerialization(wom, compressionFormat, serializationFormat, compact, pretty);
timeCompressedDeserialization(compressed, compressionFormat, serializationFormat, compact, pretty);
}
}
}
}
}
private void saveSerialized(
String title,
SerializationFormat serializationFormat,
boolean compact,
boolean pretty,
byte[] serialized) throws IOException
{
String fname = title + "-" + serializationFormat;
if (compact)
fname += "-compact";
if (pretty)
fname += "-pretty";
fname += "." + serializationFormat.toString().toLowerCase();
FileUtils.writeByteArrayToFile(new File(fname), serialized);
}
private Artifacts timeParsing(final File file) throws LinkTargetException, IOException, EngineException
{
RunMe runMe = new RunMe()
{
@Override
public void run() throws LinkTargetException, IOException, EngineException
{
parse(file);
}
};
print(2, "Parsing", timeIt(runMe, WARM_UP, ITERATIONS));
return parse(file);
}
private byte[] timeSerialization(
final Wom3Document wom,
final SerializationFormat serializationFormat,
final boolean compact,
final boolean pretty) throws IOException, SerializationException
{
RunMe runMe = new RunMe()
{
@Override
public void run() throws IOException, SerializationException
{
serializer.serialize(wom, serializationFormat, compact, pretty);
}
};
print(8, "Serialization", timeIt(runMe, WARM_UP, ITERATIONS));
return serializer.serialize(wom, serializationFormat, compact, pretty);
}
private void timeDeserialization(
final byte[] serialized,
final SerializationFormat serializationFormat,
final boolean compact,
final boolean pretty) throws IOException, DeserializationException
{
RunMe runMe = new RunMe()
{
@Override
public void run() throws IOException, DeserializationException
{
serializer.deserialize(serialized, serializationFormat, compact);
}
};
print(8, "Deserialization", timeIt(runMe, WARM_UP, ITERATIONS));
serializer.deserialize(serialized, serializationFormat, compact);
}
private byte[] timeCompression(
final byte[] serialized,
final CompressionFormat compressionFormat) throws IOException, CompressionException
{
RunMe runMe = new RunMe()
{
@Override
public void run() throws IOException, CompressionException
{
serializer.compress(serialized, compressionFormat);
}
};
print(8, "Compression", timeIt(runMe, WARM_UP, ITERATIONS));
return serializer.compress(serialized, compressionFormat);
}
private void timeDecompression(
final byte[] compressed,
final CompressionFormat compressionFormat) throws IOException, CompressionException
{
RunMe runMe = new RunMe()
{
@Override
public void run() throws IOException, CompressionException
{
serializer.decompress(compressed, compressionFormat);
}
};
print(8, "Decompression", timeIt(runMe, WARM_UP, ITERATIONS));
serializer.decompress(compressed, compressionFormat);
}
private byte[] timeCompressedSerialization(
final Wom3Document wom,
final CompressionFormat compressionFormat,
final SerializationFormat serializationFormat,
final boolean compact,
final boolean pretty) throws IOException, CompressionException, SerializationException
{
RunMe runMe = new RunMe()
{
@Override
public void run() throws IOException, CompressionException, SerializationException
{
serializer.serializeAndCompress(wom, compressionFormat, serializationFormat, compact, pretty);
}
};
print(8, "Serialization & Compression", timeIt(runMe, WARM_UP, ITERATIONS));
return serializer.serializeAndCompress(wom, compressionFormat, serializationFormat, compact, pretty);
}
private void timeCompressedDeserialization(
final byte[] compressed,
final CompressionFormat compressionFormat,
final SerializationFormat serializationFormat,
final boolean compact,
final boolean pretty) throws IOException, DeserializationException, CompressionException
{
RunMe runMe = new RunMe()
{
@Override
public void run() throws IOException, DeserializationException, CompressionException
{
serializer.decompressAndDeserialize(compressed, compressionFormat, serializationFormat, compact);
}
};
print(8, "Decompression and deserialization", timeIt(runMe, WARM_UP, ITERATIONS));
serializer.decompressAndDeserialize(compressed, compressionFormat, serializationFormat, compact);
}
// =========================================================================
private Artifacts parse(final File file) throws LinkTargetException, IOException, EngineException
{
return wmToWom(file);
}
// =========================================================================
private void print(int indent, String header, TimingResults timings)
{
String msg = String.format(""
+ "Mean: % 7.2f ms\n"
+ "Median: % 7.2f ms\n"
+ "StdDev: % 7.2f ms",
timings.mean.evaluate() * 1000.0,
timings.median.evaluate() * 1000.0,
timings.stddev.evaluate() * 1000.0);
System.out.println(StringTools.indent(header + ":", StringTools.strrep(' ', indent)));
System.out.println(StringTools.indent(msg, StringTools.strrep(' ', indent + 2)));
}
private long getSize(byte[] bytes)
{
return bytes.length;
}
private long getSize(String str)
{
return getSize(str.getBytes(CHARSET));
}
private void printSize(int indent, String what, long bytes)
{
String msg = String.format("Size of %s: %d Bytes", what, bytes);
System.out.println(StringTools.indent(msg, StringTools.strrep(' ', indent)));
}
private void printSize(
int indent,
String what,
long bytes,
String comparedTo,
long comparedToBytes)
{
final double percent = (double) bytes / (double) comparedToBytes * 100.;
String msg = String.format(
"Size of %s: %d Bytes (% 3.1f %% of the size of %s)",
what, bytes, percent, comparedTo);
System.out.println(StringTools.indent(msg, StringTools.strrep(' ', indent)));
}
// =========================================================================
private byte[] compress(String str, CompressionFormat compressionFormat) throws IOException, CompressionException
{
return serializer.compress(str.getBytes(CHARSET), compressionFormat);
}
// =========================================================================
private TimingResults timeIt(RunMe runMe, int warmUp, int iterations)
{
timeIt(runMe, warmUp);
return timeIt(runMe, iterations);
}
private TimingResults timeIt(RunMe runMe, int iterations)
{
double[] values = new double[iterations];
for (int i = 0; i < iterations; ++i)
values[i] = timeIt(runMe);
TimingResults results = new TimingResults();
results.mean.setData(values);
results.median.setData(values);
results.stddev.setData(values);
return results;
}
private double timeIt(RunMe runMe)
{
long startTime = System.nanoTime();
try
{
runMe.run();
}
catch (Exception e)
{
throw new WrappedException(e);
}
long endTime = System.nanoTime();
return ((double) (endTime - startTime)) / 1000000000.;
}
private static interface RunMe
{
public void run() throws Exception;
}
private static final class TimingResults
{
public final Mean mean = new Mean();
public final Median median = new Median();
public final StandardDeviation stddev = new StandardDeviation();
}
}
test/ 0000775 0000000 0000000 00000000000 13352705604 0034033 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src java/ 0000775 0000000 0000000 00000000000 13352705604 0034754 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test org/ 0000775 0000000 0000000 00000000000 13352705604 0035543 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test/java sweble/ 0000775 0000000 0000000 00000000000 13352705604 0037024 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test/java/org engine/ 0000775 0000000 0000000 00000000000 13352705604 0040271 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test/java/org/sweble serialization/ 0000775 0000000 0000000 00000000000 13352705604 0043146 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test/java/org/sweble/engine LoadWomFromJsonTest.java 0000664 0000000 0000000 00000005757 13352705604 0047707 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
import java.io.File;
import java.util.List;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized.Parameters;
import org.sweble.wom3.Wom3Document;
import org.sweble.wom3.Wom3Node;
import org.sweble.wom3.swcadapter.utils.WtWom3Toolbox.Artifacts;
import com.google.gson.Gson;
import de.fau.cs.osr.utils.NamedParametrized;
import de.fau.cs.osr.utils.TestResourcesFixture;
@RunWith(value = NamedParametrized.class)
public class LoadWomFromJsonTest
extends
SerializationTestBase
{
private static final String INPUT_SUB_DIR = "/various";
private static final String FILTER_RX = ".*?\\.wikitext";
// =========================================================================
@Parameters
public static List enumerateInputs() throws Exception
{
TestResourcesFixture resources = getTestResourcesFixture();
return resources.gatherAsParameters(INPUT_SUB_DIR, FILTER_RX, false);
}
// =========================================================================
private Wom3Document wom;
private File inputFile;
// =========================================================================
public LoadWomFromJsonTest(
String title,
TestResourcesFixture resources,
File inputFile)
{
super(resources);
this.inputFile = inputFile;
}
// =========================================================================
@Before
public void before() throws Exception
{
Artifacts af = wmToWom(inputFile);
this.wom = af.womDoc;
}
@Test
public void testLoadWomFromCompactJson() throws Exception
{
doIt(true, true);
doIt(true, false);
}
@Test
public void testLoadWomFromVerboseJson() throws Exception
{
doIt(false, true);
doIt(false, false);
}
private void doIt(boolean compact, boolean pretty)
{
Gson gson = createWom3Gson(compact, pretty);
String womJson = gson.toJson(wom);
//System.out.println(womJson);
Wom3Node loadedWomJsonDoc = gson.fromJson(womJson, Wom3Node.class);
// Unpack article from document fragment
loadedWomJsonDoc = loadedWomJsonDoc.getFirstChild();
String loadedWomXml = gson.toJson(loadedWomJsonDoc);
Assert.assertEquals(womJson, loadedWomXml);
}
}
LoadWomFromXmlTest.java 0000664 0000000 0000000 00000006304 13352705604 0047523 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
import java.io.File;
import java.io.StringReader;
import java.util.List;
import org.apache.xerces.parsers.DOMParser;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized.Parameters;
import org.sweble.wom3.Wom3Document;
import org.sweble.wom3.swcadapter.utils.WtWom3Toolbox.Artifacts;
import org.sweble.wom3.util.SaxonWomTransformations;
import org.xml.sax.InputSource;
import de.fau.cs.osr.utils.NamedParametrized;
import de.fau.cs.osr.utils.TestResourcesFixture;
@RunWith(value = NamedParametrized.class)
public class LoadWomFromXmlTest
extends
SerializationTestBase
{
private static final String INPUT_SUB_DIR = "/various";
private static final String FILTER_RX = ".*?\\.wikitext";
// =========================================================================
@Parameters
public static List enumerateInputs() throws Exception
{
TestResourcesFixture resources = getTestResourcesFixture();
return resources.gatherAsParameters(INPUT_SUB_DIR, FILTER_RX, false);
}
// =========================================================================
private Wom3Document wom;
private File inputFile;
// =========================================================================
public LoadWomFromXmlTest(
String title,
TestResourcesFixture resources,
File inputFile)
{
super(resources);
this.inputFile = inputFile;
}
// =========================================================================
@Before
public void before() throws Exception
{
Artifacts af = wmToWom(inputFile);
this.wom = af.womDoc;
}
@Test
public void testLoadWomFromXml() throws Exception
{
String womXml = SaxonWomTransformations.printWom(this.wom);
//System.out.println(womXml);
StringReader reader = new StringReader(womXml);
InputSource is = new InputSource(reader);
DOMParser parser = new DOMParser();
//System.out.println(parser.getXMLParserConfiguration().getFeature(
// "http://xml.org/sax/features/namespaces"));
parser.setProperty(
"http://apache.org/xml/properties/" + "dom/document-class-name",
"org.sweble.wom3.impl.DocumentImpl");
parser.parse(is);
Wom3Document loadedWomXmlDoc = (Wom3Document) parser.getDocument();
String loadedWomXml = SaxonWomTransformations.printWom(loadedWomXmlDoc);
//System.out.println(loadedWomXml);
Assert.assertEquals(womXml, loadedWomXml);
}
}
resources/ 0000775 0000000 0000000 00000000000 13352705604 0036045 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test various/ 0000775 0000000 0000000 00000000000 13352705604 0037535 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test/resources NOTICE 0000664 0000000 0000000 00000001110 13352705604 0040432 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test/resources/various The articles in this directory are taken from the English, French and Spanish
Wikipedia and are licensed under the Attribution-ShareAlike 3.0 Unported
(CC BY-SA 3.0) license.
To view a copy of this license, visit
http://creativecommons.org/licenses/by-sa/3.0/
or send a letter to
Creative Commons,
171 Second Street, Suite 300,
San Francisco, California, 94105, USA.
The following articles were taken from the English Wikipedia:
"Germany", Revision: 417491803
http://en.wikipedia.org/w/index.php?oldid=417491803
Expansion is done using the script bin/enwp-expand.php.
dogs.wikitext 0000664 0000000 0000000 00000000010 13352705604 0042252 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test/resources/various [[Dog]]s raw-Germany.wikitext 0000664 0000000 0000000 00000465011 13352705604 0043527 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test/resources/various {{Dablink|This article is about the country. For other uses of terms redirecting here, see [[Germany (disambiguation)]] and [[Deutschland (disambiguation)]].}}
{{Use dmy dates|date=January 2011}}
{{pp-semi-protected|small=yes}}
{{pp-move-indef}}
{{Featured article}}
{{Infobox country
|native_name={{lang|de|''Bundesrepublik Deutschland''}}
|conventional_long_name=Federal Republic of Germany
|common_name=Germany
|national_anthem=[[File:Deutschlandlied played by USAREUR Band.ogg]]
The third stanza of {{lang|de|''[[Deutschlandlied|Das Lied der Deutschen]]''}}
|image_flag=Flag of Germany.svg
|image_coat=Coat of Arms of Germany.svg
|image_map=EU-Germany.svg
|map_caption={{map_caption|location_color=dark green|region=Europe|region_color=dark grey|subregion=the [[European Union]]|subregion_color=green|legend=EU-Germany.svg}}
|official_languages=[[German language|German]] {{Ref|1}}
|demonym=German
|capital=[[Berlin]]
|latd=52|latm=31|latNS=N|longd=13|longm=23|longEW=E
|largest_city=capital
|government_type=[[Federal republic|Federal]] [[parliamentary republic]]
|leader_title1=[[President of Germany|President (Head of State)]]
|leader_name1=[[Christian Wulff]] ([[Christian Democratic Union (Germany)|CDU]])
|leader_title2=[[Chancellor of Germany (Federal Republic of Germany)|Chancellor (Head of Government)]]
|leader_name2=[[Angela Merkel]] ([[Christian Democratic Union (Germany)|CDU]])
|leader_title3=[[President of the Bundestag]]
|leader_name3=[[Norbert Lammert]] ([[Christian Democratic Union (Germany)|CDU]])
|leader_title4=[[President of the Bundesrat]]
|leader_name4=[[Hannelore Kraft]] ([[Social Democratic Party of Germany|SPD]])
|sovereignty_type=[[Kingdom of Germany|Formation]]
|sovereignty_note=
|established_event1=[[Holy Roman Empire]]
|established_date1=2 February 962
|established_event2=[[Unification of Germany|Unification]]
|established_date2=18 January 1871
|established_event3=[[History of Germany|Federal Republic]]
|established_date3=23 May 1949
|established_event4=[[German reunification|Reunification]]
|established_date4=3 October 1990
|accessionEUdate=25 March 1957
|EUseats=99
|area_km2=357,021
|area_sq_mi=137,847
|area_rank=63rd
|area_magnitude=1 E11
|percent_water=2.416
|population_estimate = 81,757,600
|population_estimate_year = 2010
|population_estimate_rank = 15th
|population_census =
|population_census_year =
|population_estimate_year = Jan. 1, 2010
|population_estimate_rank = 15th
|population_density_km2 = 229
|population_density_sq_mi = 593
|population_density_rank = 55th
|GDP_PPP_year = 2010
|GDP_PPP = $2.932 trillion{{cite encyclopedia|url=http://www.imf.org/external/pubs/ft/weo/2010/02/weodata/weorept.aspx?pr.x=61&pr.y=14&sy=2008&ey=2015&scsm=1&ssd=1&sort=country&ds=.&br=1&c=134&s=NGDP_R%2CNGDP_RPCH%2CNGDP%2CNGDPD%2CNGDP_D%2CNGDPRPC%2CNGDPPC%2CNGDPDPC%2CNGAP_NPGDP%2CPPPGDP%2CPPPPC%2CPPPSH%2CPPPEX%2CNID_NGDP%2CNGSD_NGDP&grp=0&a=|title=Germany|publisher=International Monetary Fund|accessdate=20 February 2011}}
|GDP_PPP_rank=
|GDP_PPP_per_capita=$35,930
|GDP_PPP_per_capita_rank=
|GDP_nominal=$3.305 trillion
|GDP_nominal_rank=
|GDP_nominal_year=2010
|GDP_nominal_per_capita=$40,511
|GDP_nominal_per_capita_rank=
|HDI_year=2010
|HDI={{increase}} 0.885{{cite web|url=http://hdr.undp.org/en/media/HDR_2010_EN_Table1.pdf|title=Human development indices|publisher=[[United Nations Development Programme]]|accessdate=4 November 2010}}
|HDI_rank=10th
|HDI_category=very high
|Gini=27 {{Update after|2011|04|reason=Regularly check CIA factbook or http://www.wider.unu.edu/research/Database/en_GB/wiid/ }}
|Gini_year=2006
|Gini_category=low
|currency=[[Euro]] ([[Euro sign|€]]){{Ref|2}}(2002 – present)
|currency_code=EUR
|time_zone=CET
|utc_offset=+1
|time_zone_DST=CEST
|utc_offset_DST=+2
|drives_on=right
|cctld= [[.de]] {{Ref|3}}
|calling_code=[[Telephone numbers in Germany|49]]
|ISO_3166-1_alpha2=DE
|ISO_3166-1_alpha3=DEU
|ISO_3166-1_numeric=?
|alt_sport_code=GER
|vehicle_code=D
|aircraft_code=D
|footnote1={{note|1}} Danish, [[Low German]], [[Sorbian languages|Sorbian]], [[Romani language|Romany]], and [[Frisian languages|Frisian]] are officially recognised and protected by the [[European Charter for Regional or Minority Languages|ECRML]].
|footnote2= {{note|2}} Before 2002: [[Deutsche Mark]] (DEM).
|footnote3={{note|3}} Also [[.eu]], shared with other [[European Union]] member states.
}}
'''Germany''' ({{IPAc-en|En-uk-Germany.ogg|ˈ|dʒ|ɜr|m|ən|i}}), officially the '''Federal Republic of Germany''' ({{lang-de|Bundesrepublik Deutschland}}, {{IPA-de|ˈbʊndəsʁepuˌbliːk ˈdɔʏtʃlant|pron|De-Bundesrepublik_Deutschland.ogg}}),{{cite book |editor=Max Mangold (ed.)|title=Duden, Aussprachewörterbuch (Duden Pronunciation Dictionary) |edition=6th |year=1995 |publisher=Dudenverlag Bibliographisches Institut & F.A. Brockhaus |location=Mannheim |language=German |isbn=3-411-04066-1|pages=271, 53f}}The country's long name in its [[Languages of Germany|regional languages]] include:
*{{lang-dsb|Nimska zwězkowa republika}};
*{{lang-hsb|Němska zwjazkowa republika}};
*{{lang-da|Forbundsrepublikken Tyskland}};
*{{lang-nds| Bundsrepubliek Düütschland}};
*{{lang-rmy|I Federalni Republika Jermaniya}};
*[[Frisian language|Frisian]]: Bûnsrepublyk Dútslân;
*[[Palatinate German]]: Bundesrepublik vun Daitschlond is a country in [[Western Europe|Western]] and [[Central Europe]]. It is bordered to the north by the [[North Sea]], [[Denmark]], and the [[Baltic Sea]]; to the east by [[Poland]] and the [[Czech Republic]]; to the south by [[Austria]] and [[Switzerland]]; and to the west by [[France]], [[Luxembourg]], [[Belgium]], and the [[Netherlands]]. The territory of Germany covers an area of 357,021 km2 and is influenced by a [[temperate climate|temperate seasonal climate]]. With 81.8 million inhabitants, it is the most populous member state of the [[European Union]],{{cite web |url=http://epp.eurostat.ec.europa.eu/cache/ITY_OFFPUB/KS-QA-09-047/EN/KS-QA-09-047-EN.PDF |title=First demographic estimates for 2009 |publisher=[[Eurostat]] |accessdate=31 January 2010}} and home to the [[List of countries by immigrant population|third-largest number of international migrants]] worldwide.{{cite web |url=http://www.migrationinformation.org/datahub/countrydata/country.cfm |title=Germany: Inflow of foreign population by country of nationality, 1994 to 2003 | publisher = Migrationinformation.org |accessdate=4 January 2010}}
A region named [[Germania]], inhabited by several [[Germanic people]]s, was [[Germania (book)|documented]] before AD 100. Beginning in the 10th century, German territories formed a central part of the [[Holy Roman Empire]], which lasted until 1806. During the 16th century, northern German regions became the centre of the [[Protestant Reformation]] while southern and western parts remained dominated by [[Roman Catholic]] denominations. As a modern [[nation-state]], the country was first [[Unification of Germany|unified]] in 1871, with the inception of the [[German Empire]]. After the proclamation of the [[Weimar Republic]] in 1918 the [[Third Reich]] followed in 1933. The period was marked by a dictatorship and the initiation of [[World War II]]. After 1945, Germany was divided by [[Allied Occupation Zones in Germany|allied occupation]], and evolved into two states, [[East Germany]] and [[West Germany]]. In 1990 Germany was [[German reunification|reunified]]. West Germany was a founding member of the [[European Communities|European Community]] in 1957, which became the EU in 1993. It is part of the [[Schengen Area]] and since 1999 a member the [[eurozone]].{{cite news |title=On Jan. 1, out of many arises one Euro |newspaper=St. Petersburg Times |location= Florida, USA |first= Susan | last = Taylor Martin |date=28 December 1998 |page=National, 1.A }}{{Cite news|title = Germans Say Goodbye to the Mark, a Symbol of Strength and Unity |newspaper=The New York Times| accessdate = 26 April 2010| url = http://www.nytimes.com/2002/01/01/world/germans-say-goodbye-to-the-mark-a-symbol-of-strength-and-unity.html?pagewanted=1 |first=Edmund L. | last = Andrews |date=1 January 2002 }}{{Cite news |title = Euro notes and coins to flood continent |first=David | last = Smith |date=15 June 2001 |newspaper=New Straits Times| location = Kuala Lumpur | accessdate = 26 April 2010| url = http://news.google.co.uk/newspapers?id=liwhAAAAIBAJ&sjid=j3gFAAAAIBAJ&pg=1283,3813941&dq=germany+euro+currency+introduced+1999&hl=en }}
Germany is a [[Federation|federal]] [[parliamentary republic]] of [[States of Germany|sixteen states]]. The capital and largest city is [[Berlin]]. Germany is a member of the [[United Nations]], [[NATO]], the [[G8]], the [[G-20 major economies|G20]], the [[OECD]] and assumed a nonpermanent seat on the [[UN Security Council]] for the 2011–12 term. It is a [[Great power#Aftermath of the Cold War|major power]] with the world's [[List of countries by GDP (nominal)|fourth largest economy]] by nominal [[GDP]] and the [[List of countries by GDP (PPP)|fifth largest]] by [[purchasing power parity]]. It is the [[List of countries by exports|second largest exporter]] and [[List of countries by imports|third largest importer]] of goods. In absolute terms, Germany allocates the third biggest annual [[development aid]] budget in the world,{{cite news | url = http://www.topnews.in/germany-worlds-second-biggest-aid-donor-after-us-229970 | title = Germany world's second biggest aid donor after US | work = TopNews India | accessdate = 10 April 2008 | agency = DPA | date = 4 April 2008 }} while its [[List of countries by military expenditures|military expenditure ranked seventh]].{{cite web |url=http://www.sipri.org/contents/milap/milex/mex_major_spenders.pdf/download |archiveurl=http://web.archive.org/web/20070814224502/http://www.sipri.org/contents/milap/milex/mex_major_spenders.pdf/download |archivedate=14 August 2007 |title=The fifteen major spenders in 2006 |accessdate=23 August 2007 |year=2007 |format=PDF |work=Recent trends in military expenditure| publisher=Stockholm International Peace Research Institute}} The country has developed a [[List of countries by Human Development Index|very high standard of living]] and a comprehensive system of [[social security]]. It holds a key position in European affairs and maintains a multitude of close partnerships on a global level.{{cite news | url=http://www.iht.com/articles/2008/04/04/europe/poll.php | title = The leader of Europe? Answers an ocean apart | newspaper = International Herald Tribune | date = 4 April 2008 | accessdate =4 April 2008 | location = Paris }} Germany is recognised as a scientific and technological leader in several fields.[http://www.innovations-report.de/html/berichte/informationstechnologie/bericht-109339.html Confidently into the Future with Reliable Technology] www.innovations-report.de. 7 May 2008. Retrieved 4 April 2008.
==History==
{{Main|History of Germany}}
===Etymology===
{{Main|Names of Germany|List of country name etymologies}}
The English word "Germany" derives from the [[Greek language|Greek]] {{cite book|last = Stümpel|first = Gustav|title = Name und Nationalität der Germanen. Eine neue Untersuchung zu Poseidonios, Caesar und Tacitus|origyear = 1932|publisher = Dieterich|location = Leipzig|language = German|pages = 60}} term [[Germania]]. The name "Germania" came into use after [[Julius Caesar]] adopted it from a [[Gaulish language|Gallic]] term for the peoples east of the [[Rhine]] that could possibly have meant "neighbour", or "men of forests", or even "men with spears".{{cite book |last= Schulze |first=Hagen |authorlink=Hagen Schulze |title=Germany: A New History |publisher= [[Harvard University Press]]|page=4 |year=1998 |isbn= 0-674-80688-3}}{{cite book | url=http://www.oxfordreference.com/views/ENTRY.html?subview=Main&entry=t27.e6407 | section = German | title = The Concise Oxford Dictionary of English Etymology | editor = Hoad, T.F. | publisher = Oxford University Press | year = 1996 | accessdate = 4 March 2008 }}{{cite web|url=http://www.zeno.org/Meyers-1905/A/G%C4%93r |title=Meyers Encyclopedia:Ger |publisher=Zeno.org |accessdate=26 September 2010}}
===Germanic tribes 750 BC–260===
{{Main|Germanic peoples|Germania}}
[[File:Germanic tribes (750BC-1AD).png|thumb|left|Expansion of the [[Germanic peoples|Germanic tribes]] 750 BC – AD 1]]
The [[ethnogenesis]] of the [[Germanic peoples|Germanic tribes]] is assumed to have occurred during the [[Nordic Bronze Age]], or at the latest during the [[Pre-Roman Iron Age]]. From southern Scandinavia and northern Germany, the tribes began expanding south, east and west in the 1st century BC, coming into contact with the [[Celt]]ic tribes of [[Gaul]] as well as [[Iranian peoples|Iranian]], [[Balts|Baltic]], and [[Slavic peoples|Slavic]] tribes in [[Eastern Europe]]. Little is known about early Germanic history, except through their recorded interactions with the [[Roman Empire]], etymological research and archaeological finds.{{cite book | first = Jill N. | last = Claster | title = Medieval Experience: 300–1400 | publisher = [[New York University Press]] | year = 1982 | page = 35 | isbn=0-8147-1381-5 }}
Under [[Augustus]], the Roman General [[Publius Quinctilius Varus]] began to invade Germania (a term used by the Romans to define a territory running roughly from the [[Rhine]] to the [[Ural Mountains]]), and it was in this period that the Germanic tribes became familiar with Roman tactics of warfare while maintaining their tribal identity. In AD 9, three [[Roman legion]]s led by Varus were defeated by the [[Cheruscan]] leader [[Arminius]] in the [[Battle of the Teutoburg Forest]]. Modern Germany, as far as the [[Rhine]] and the [[Danube]], thus remained outside the Roman Empire. By AD 100, the time of [[Tacitus]]' ''[[Germania (book)|Germania]]'', Germanic tribes settled along the Rhine and the Danube (the [[Limes Germanicus]]) , occupying most of the area of modern Germany; Austria, southern [[Bavaria]] and the western [[Rhineland]], however, were Roman provinces. The 3rd century saw the emergence of a number of large West Germanic tribes: [[Alamanni]], [[Franks]], [[Chatti]], [[Saxons]], [[Frisians]], [[Sicambri]], and [[Thuringii]]. Around 260, the Germanic peoples broke through the Limes and the Danube frontier into Roman-controlled lands.{{cite book | series= The Cambridge Ancient History | title = The crisis of empire, A.D. 193–337 | volume = 12 | page = 442 | isbn=0-521-30199-8 | first = Alan K. | last = Bowman | first2 = Peter | last2 = Garnsey | first3 = Averil | last3 = Cameron | publisher = Cambridge University Press | year = 2005}}
===Migration Period and Frankish Empire 260-800===
{{Main|Migration Period|Franks}}
After the invasion of the [[Huns]] 375 and with the decline of Rome from 395 began the great migration, during which the Germanic tribes moved further and further to the southwest. Simultaneously several large tribes formed on the territory of modern Germany and displaced the smaller Germanic tribes. These were the [[Franks]] and [[Alamanni]] in the West, the [[Saxons]] in the North, the [[Bavarians]] in the South and the [[Thuringians]] in the East who walked into the nearly deserted areas of today's East Germany in the 7th Century to the Elbe-Saale line which marked the new border to the Slavic tribes. Large segments of the population of today's East Germany were characterized as Slavic (Germania Slavica) in the Early Middle Age. Only during the High Middle Ages, they were assimilated ([[Ostsiedlung]]). The main part of Western and Central Europe and thus also large areas of today's South and West Germany were occupied by the Franks, now Northern Germany was ruled by the Saxons and Slavs. All areas of the former [[Frankish Empire]] associated with todays Germany were in the eastern part of the empire, which had been known since the [[Merovingian]] period as [[Austrasia]]. A certain self-rule of Austrasia appeared the first time in 614 as King [[Clotaire II]] had to sign the ''Edictum Chlotharii'', in which it was confirmed that each of the three Frankish sub-kingdoms (Neustria, Austrasia, Burgundy) came under the rule of a [[Mayor of the Palace]].
===Holy Roman Empire 962–1806===
{{Main|Holy Roman Empire}}
{{See also|Medieval demography|Ostsiedlung}}
[[File:Weltliche Schatzkammer Wien (169)pano.jpg|thumb|upright|The [[Imperial Crown of the Holy Roman Empire|Imperial Crown]] of the [[Holy Roman Empire]], created around AD 1000]]
On 25 December 800, [[Charlemagne]] founded the [[Carolingian Empire]], which was [[Treaty of Verdun|divided in 843]]. The medieval empire resulted from the [[East Francia|eastern portion]] of this division and existed in varying forms from 962 until 1806. Its territory stretched from the [[Eider River]] in the north to the [[Mediterranean Sea|Mediterranean]] coast in the south. Often referred to as the [[Holy Roman Empire]] (or the Old Empire), it was officially called the ''Sacrum Romanum Imperium Nationis Germanicæ'' (Holy Roman Empire of the German Nation) starting in 1448, to adjust the title to its then reduced territory.
[[File:Martin Luther by Lucas Cranach der Ältere.jpeg|thumb|left|150px|[[Martin Luther]] initiated the [[Protestant Reformation]].]]
Under the reign of the [[Ottonian]] emperors (919–1024), the [[Stem duchy|duchies]] of [[Duchy of Lorraine|Lorraine]], [[Duchy of Saxony|Saxony]], [[Franconia]], [[Swabia]], [[Thuringia]], and [[Bavaria]] were consolidated, and the German king was crowned [[Holy Roman Emperor]] of these regions in 962. Under the reign of the [[Salian]] emperors (1024–1125), the Holy Roman Empire absorbed northern Italy and [[Burgundy (region)|Burgundy]], although the emperors lost power through the [[Investiture Controversy]]. Under the [[House of Hohenstaufen|Hohenstaufen]] emperors (1138–1254), the German princes increased their influence further south and east into territories inhabited by [[Slavic peoples|Slavs]], preceding [[History of German settlement in Eastern Europe|German settlement]] in these areas and further east ''([[Ostsiedlung]])''. Northern German towns grew prosperous as members of the [[Hanseatic League]]. Starting with the [[Great Famine of 1315–1317|Great Famine]] in 1315, then the [[Black Death]] of 1348–50, the population of Germany plummeted.{{ cite book | url=http://www.vlib.us/medieval/lectures/black_death.html | title = The Great Famine (1315–1317) and the Black Death (1346–1351) | first = Lynn Harry | last = Nelson | publisher = [[University of Kansas]] }}
The edict of the [[Golden Bull of 1356|Golden Bull]] in 1356 provided the basic constitution of the empire that lasted until its dissolution. It codified the election of the emperor by seven [[prince-elector]]s who ruled some of the most powerful principalities and archbishoprics. Beginning in the 15th century, the emperors were elected nearly exclusively from the [[Habsburg]] dynasty of [[Archduchy of Austria|Austria]].
The monk [[Martin Luther]] publicised his [[The 95 Theses|95 Theses]] in 1517, challenging practices of the [[Roman Catholic Church]], initiating the [[Protestant Reformation]]. A separate [[Lutheranism|Lutheran]] church became the official religion in many German states after 1530. Religious conflict led to the [[Thirty Years' War]] (1618–1648), which devastated German lands.Rempel, Gerhard. [http://mars.wnec.edu/~grempel/courses/wc2/lectures/30yearswar.html The Thirty-Years-War]. Western [[New England College]]. The population of the German states was reduced by about 30%.{{cite book | url=http://books.google.com/?id=eGsCGAdH4YQC&pg=PA51&dq#v=onepage&q=&f=false | title = The savage wars of peace: England, Japan and the Malthusian trap | first = Alan | last = Macfarlane | year = 1997 | page=51 |isbn = 0-631-18117-2 }} The [[Peace of Westphalia]] (1648) ended religious warfare among the German states, but the empire was ''de facto'' divided into numerous independent principalities. From 1740 onwards, the [[German dualism|dualism]] between the Austrian [[Habsburg Monarchy]] and the [[Kingdom of Prussia]] dominated German history. In 1806, the ''Imperium'' was overrun and dissolved as a result of the [[Napoleonic Wars]].{{cite book | last = Fulbrook | first = Mary | title = A Concise History of Germany | publisher = Cambridge University Press | year = 1991 | page = 97 | isbn = 0-521-54071-2 }}
===Restoration and revolution 1814–1871===
{{Main|German Confederation}}
[[File:Nationalversammlung.jpg|thumb|[[Frankfurt parliament]] in 1848]]
Following the fall of [[Napoleon I of France|Napoleon Bonaparte]], the [[Congress of Vienna]] convened in 1814 and founded the [[German Confederation]] (Deutscher Bund), a loose league of [[List of German Confederation member states|39 sovereign states]]. Disagreement with [[European Restoration|restoration]] politics partly led to the rise of [[Liberalism in Germany|liberal]] movements, demanding unity and freedom. These, however, were followed by new measures of repression by the Austrian statesman [[Klemens Wenzel von Metternich|Metternich]]. The ''[[Zollverein]]'', a tariff union, profoundly furthered economic unity in the German states. During this era many [[Germans]] had been stirred by the ideals of the [[French Revolution]], and [[nationalism]] became a more significant force, especially among young intellectuals. For the first time, the colours of black, red and gold were chosen to represent the movement, which later became the [[Flag of Germany|national colours]].Martin, Norman. [http://www.fotw.net/flags/de1848.html German Confederation 1815–1866 (Germany)]. Flags of the World. 5 October 2000. Retrieved 7 December 2006.
In the light of a [[Revolutions of 1848|series of revolutionary movements in Europe]], which successfully established a republic [[Revolutions of 1848 in France|in France]], intellectuals and commoners started the [[Revolutions of 1848 in the German states]]. The monarchs initially yielded to the revolutionaries' liberal demands. King [[Frederick William IV of Prussia]] was offered the title of [[Emperor]], but with a loss of power; he rejected the crown and the proposed constitution, leading to a temporary setback for the movement. Conflict between King [[William I, German Emperor|William I]] of [[Prussia]] and the increasingly liberal parliament erupted over military reforms in 1862, and the king appointed [[Otto von Bismarck]] the new [[Prime Minister of Prussia]]. Bismarck successfully waged [[Second War of Schleswig|war on Denmark]] in 1864. Prussian victory in the [[Austro-Prussian War]] of 1866 enabled him to create the [[North German Federation]] (Norddeutscher Bund) and to exclude [[Austrian Empire|Austria]], formerly the leading German state, from the affairs of the remaining German states.
===German Empire 1871–1918===
{{Main|German Empire|World War I}}
[[File:Wernerprokla.jpg|thumb|right|Foundation of modern Germany in [[Versailles]], France, 1871. [[Otto von Bismarck|Bismarck]] is at the centre in a white uniform.]]
The state known as ''Germany'' was [[political union|unified]] as a modern nation-state in 1871, when the [[German Empire]] was forged, with the [[Kingdom of Prussia]] as its largest constituent.
After the French defeat in the [[Franco-Prussian War]], the German Empire was proclaimed in [[Palace of Versailles|Versailles]] on 18 January 1871. The [[House of Hohenzollern|Hohenzollern]] dynasty of Prussia ruled the new empire, whose capital was Berlin. The empire was a unification of all the scattered parts of Germany except Austria ({{lang|de|[[Kleindeutsche Lösung|''Kleindeutschland'']]}}, or "Lesser Germany"). But internally the official political unification came rather sequentially: Germany had no national flag until 1892 and no national hymn until after WW I. Beginning in 1884, Germany began establishing [[List of former German colonies|several colonies]] outside Europe.
[[File:Map-DR-Prussia.svg|left|thumb|Imperial Germany (1871–1918), with the dominant [[Kingdom of Prussia]] in blue]]
In the {{lang|de|''[[Gründerzeit]]''}} period following the [[unification of Germany]], Emperor [[William I, German Emperor|William I's]] foreign policy secured Germany's position as a great nation by forging alliances, isolating [[French Third Republic|France]] by diplomatic means, and avoiding war. Under [[William II, German Emperor|William II]], however, Germany, [[New Imperialism|like other European powers]], took an [[imperialism|imperialistic]] course leading to friction with neighbouring countries. Most alliances in which Germany had been previously involved were not renewed, and new alliances excluded the country. Specifically, France established new relationships by signing the [[Entente Cordiale]] with Great Britain and securing ties with the [[Russian Empire]]. Aside from its contacts with [[Austria-Hungary]], Germany became increasingly isolated.
Germany's imperialism reached outside its own country, and it joined many other powers in Europe in claiming its share of Africa. The [[Berlin Conference]] divided Africa between the European powers. Germany owned several pieces of land in Africa including [[German East Africa]], [[German South-West Africa|South-West Africa]], [[Togoland|Togo]], and [[Cameroon]]. The [[Scramble for Africa]] caused tension between the [[great power]]s that may have contributed to the conditions that led to [[World War I]].
The [[Assassination in Sarajevo|assassination]] of [[Archduke Franz Ferdinand of Austria|Austria's crown prince]] on 28 June 1914 triggered [[World War I]]. Germany, as part of the unsuccessful [[Central Powers]], suffered defeat against the [[Allies of World War I|Allied Powers]] in [[World War I casualties|one of the bloodiest]] conflicts of all time. An estimated two million German soldiers died in World War I.{{cite news | url = http://www.spiegel.de/international/germany/0,1518,530319,00.html | title = Last German World War I Veteran Believed to Have Died | newspaper = Spiegel Online | date = 22 January 2008 | first = David | last = Crossland }} The [[German Revolution]] broke out in November 1918, and Emperor William II and all German ruling princes [[abdication|abdicated]]. An [[Armistice with Germany (Compiègne)|armistice]] putting an end to the war was signed on 11 November, and Germany was forced to sign the [[Treaty of Versailles]] in June 1919. Its negotiation, contrary to traditional post-war diplomacy, excluded the defeated Central Powers. The treaty was perceived in Germany as a humiliating continuation of the war. The treaty's harshness is often cited as an influence toward the later rise of [[Nazism]] in the country.{{cite book | first = Stephen J. | last = Lee | title = Europe, 1890–1945 | publisher = Routledge | year = 2003 | page = 131 | isbn = 0-415-25455-8 | location = London}}
===Weimar Republic 1919–1933===
{{Main|Weimar Republic}}
[[File:Ausrufung Republik Scheidemann.jpg|upright|thumb|[[Philipp Scheidemann]] proclaiming the German Republic in 1918. The [[Weimar Republic|Weimar]] era was dominated by political unrest.]]
At the beginning of the German Revolution, Germany was declared a [[republic]] and the monarchy collapsed. However, the struggle for power continued, with radical-left communists [[Bavarian Socialist Republic|seizing power in Bavaria]], but failing to take control of all of Germany. The revolution came to an end in August 1919, when the [[Weimar Republic]] was formally established. The [[Weimar Constitution]] came into effect with its signing by [[President of Germany|President]] [[Friedrich Ebert]] on 11 August 1919.
Suffering from the [[Great Depression]], the harsh peace conditions dictated by the [[Treaty of Versailles]], and a long succession of more or less unstable governments, the people of Germany increasingly lacked identification with their political system and the "Establishment Parties" in their [[parliamentary democracy]]. This was exacerbated by a widespread right-wing ([[monarchism|monarchist]], ''[[Völkisch movement|völkisch]]'', and Nazi) {{lang|de|''[[Dolchstoßlegende]]''}}, which promoted the view that Germany had lost World War I because of the efforts and influence of those who wanted to overthrow the government. The top brass of the Weimar government was accused of betraying the German Nation by signing the Versailles Treaty, while the radical left-wing [[communism|communists]], such as the [[Spartacist League]], had wanted a revolution to abolish "[[capitalism|capitalist rule]]" in favour of a {{lang|de|''[[Council communism|Räterepublik]]''}}, and were also targeted.
Discontentment with the new Weimar government helped fuel the growth of the [[Communist Party of Germany|German Communist Party]]. Many conservatives were drawn towards the reactionary/revolutionary right, particularly the National Socialist German Workers Party—the [[Nazi Party]]. By 1932, these two parties controlled the majority of parliament (296 total parliamentary seats by July 1932). After a series of unsuccessful cabinets, President [[Paul von Hindenburg]] made a crucial decision: on 30 January 1933, seeing little alternative and pushed by right-wing advisors, von Hindenburg appointed [[Adolf Hitler]] as [[Chancellor of Germany (German Reich)|Chancellor of Germany]], honouring Hitler's request.
===Third Reich 1933–1945===
{{Main|Nazi Germany}}
[[File:Bundesarchiv Bild 183-S33882, Adolf Hitler retouched.jpg|thumb|upright|[[Adolf Hitler]], chancellor of the ''[[Nazi Germany|Großdeutsches Reich]]'']]
On 27 February 1933 the [[Reichstag fire|Reichstag]] building went up in flames, and a consequent [[Reichstag Fire Decree|emergency decree]] abrogated basic citizens' rights. An [[Enabling Act]] passed in parliament gave Hitler unrestricted legislative power. Only the [[Social Democratic Party of Germany|Social Democratic Party]] voted against it, while [[KPD|Communist]] [[Member of Parliament|MPs]] had already been imprisoned.{{cite web |url=http://www.dhm.de/lemo/html/nazi/innenpolitik/ermaechtigungsgesetz/index.html |title=Das Ermächtigungsgesetz 1933 |accessdate=12 September 2008 |publisher=[[Deutsches Historisches Museum]]| language=German |quote=An der Abstimmung nicht teilnehmen konnten die 81 Abgeordneten der Kommunistischen Partei Deutschlands (KPD). Ihre Mandate waren auf Basis der Reichstagsbrandverordnung bereits am 8. März 1933 annulliert worden. }}
Stackelberg, Roderick (1999). ''Hitler's Germany: origins, interpretations, legacies''. Routledge. p. 103. ISBN 0-415-20114-4.
Scheck, Raffael. [http://www.colby.edu/personal/r/rmscheck/GermanyE1.html Establishing a Dictatorship: The Stabilization of Nazi Power] Colby College. Retrieved 12 July 2006. Using his powers to crush any actual or potential resistance, Hitler established a centralised [[totalitarian state]] within months. Industry was revitalised with a focus on military rearmament.{{cite web |url=http://www.dhm.de/lemo/html/nazi/wirtschaft/index.html|title=Industrie und Wirtschaft |accessdate=12 September 2008 |publisher=Deutsches Historisches Museum |language=German |quote=Der Vierjahresplan sollte ab 1936 die wirtschaftliche Kriegsfähigkeit Deutschlands herbeiführen. . . .Bereits im Februar 1933 erklärte Hitler, dass alle öffentlichen Maßnahmen zur Arbeitsbeschaffung zugleich der "Wehrhaftmachung" zu dienen hätten und den Interessen des Staates untergeordnet seien. . . .}} In 1935, Germany reacquired control of the [[Saar (League of Nations)|Saar]] and in 1936 military control of the [[Rhineland]], both of which had been lost in the [[Treaty of Versailles]].
Leading to [[World War II]] and roughly in parallel with military rearmament, German foreign policy became more aggressive and [[Lebensraum|expansionistic]]. In 1938 and 1939, [[Anschluss|Austria]] and [[German occupation of Czechoslovakia|Czechoslovakia]] were brought under control and the [[invasion of Poland]] prepared through the [[Molotov–Ribbentrop pact]] and [[Operation Himmler]]. On 1 September 1939 the German [[Wehrmacht]] launched a [[blitzkrieg]] on [[Second Polish Republic|Poland]], which was swiftly occupied by Germany and by the Soviet [[Red Army]]. The UK and France declared war on Germany, marking the beginning of World War II in Europe. As the war progressed, Germany and its [[Axis powers|allies]] quickly gained control of much of [[Occupied Europe|continental Europe]] while the plan to [[Operation Sea Lion|occupy the United Kingdom]] failed.
[[File:Potsdamer Platz 1945.jpg|thumb|left|Berlin in ruins after [[World War II]], [[Potsdamer Platz]] 1945]]On 22 June 1941, Germany broke the [[Molotov–Ribbentrop pact]] and [[Operation Barbarossa|invaded the Soviet Union]]. The same year, [[Empire of Japan|Japan]] [[Attack on Pearl Harbor|attacked]] the American base at Pearl Harbor, and Germany declared war on the United States as a consequence of its alliance with Japan. Although the German army advanced into the Soviet Union quite rapidly, the [[Battle of Stalingrad]] marked a major turning point in the war. Subsequently, the German army started to retreat on the [[Eastern Front (World War II)|Eastern front]]. In September 1943, Germany's ally Italy surrendered, and German forces were forced to defend an additional front in Italy. [[Normandy Landings|D-Day]] marked another major turning point in the war, opening up a [[Western Front (World War II)|Western front]]; the Allied forces landed on the beaches of [[Normandy]] and made advances towards German territory. Germany's defeat soon followed. On 8 May 1945, [[Victory in Europe Day|the German armed forces surrendered]] after the [[Red Army]] occupied Berlin. Approximately seven million German soldiers and civilians – including [[ethnic Germans]] from Eastern Europe – died during World War II.Steinberg, Heinz Günter. ''Die Bevölkerungsentwicklung in Deutschland im Zweiten Weltkrieg: mit einem Überblick über die Entwicklung von 1945 bis 1990.'' Bonn 1991.
In what later became known as [[The Holocaust]], the Third Reich regime enacted governmental policies directly subjugating many dissidents and minorities. About seventeen million people were murdered by the Nazis during the Holocaust, including six million [[Jews]] and a sizable number of [[Porajmos|Gypsies]], [[Jehovah's Witnesses]], [[Generalplan Ost|Poles and other Slavs]], including [[Nazi crimes against Soviet POWs|Soviet POWs]], [[Action T4|people with mental and/or physical disabilities]], homosexuals, and members of the political opposition.Niewyk, Donald L.; Nicosia, Francis R. [http://books.google.ca/books?id=lpDTIUklB2MC&pg=PP1&dq=Niewyk,+Donald+L.+The+Columbia+Guide+to+the+Holocaust&sig=4igufxQHRCNrkjwRuMt1if_mf5M#PPA45,M1 ''The Columbia Guide to the Holocaust''], [[Columbia University Press]], 2000, pp. 45–52. World War II and the Nazi genocide were responsible for more than 40 million dead in Europe.{{cite news | url = http://news.bbc.co.uk/2/hi/europe/4530565.stm | title = Leaders mourn Soviet wartime dead | work = BBC News Online |location = London | date = 9 May 2005 }} The [[Nuremberg trials]] of [[Nazi war criminals]] were held after World War II."[http://www.bbc.co.uk/history/worldwars/wwtwo/nuremberg_article_01.shtml Nuremberg: Nazis On Trial]". Professor Richard Overy. BBC History.
{{Clear}}
===Division and reunification 1945–1990===
{{Main|History of Germany (1945–1990)}}
[[File:Map-Germany-1945.svg|thumb|Post-WWII [[Allied Occupation Zones in Germany|occupation zones]] of Germany, in its 1937 borders, with territories east of the [[Oder-Neisse line]] shown as annexed by Poland and the Soviet Union, plus the [[Saar (protectorate)|Saar protectorate]] and divided Berlin. [[East Germany]] was formed by the Soviet Zone, while West Germany was formed by the American, British, and French zones in 1949 and the Saar in 1957.]]
The war resulted in the [[World War II casualties|death of over seven million German soldiers and civilians]]; [[Oder-Neisse line|large territorial losses]]; [[Expulsion of Germans after World War II|the expulsion of about 15 million Germans]] from [[Former eastern territories of Germany|the eastern areas of Germany]] and other countries; [[Rape during the occupation of Germany|mass rape of German women]];{{cite book | last = Beevor | first = Antony | authorlink = Antony Beevor | title = Berlin: The downfall 1945 | origyear = 2002 | year = 2003 | publisher = Penguin | location = London | isbn =0-14-028696-9 | pages = 409–412 | quote = Altogether at least 2 million German women are thought to have been raped . . . }}
{{cite book | last = Beevor | first = Antony | authorlink = Antony Beevor | title = Berlin: The downfall 1945 | origyear = 2002 | year = 2003 | publisher = Penguin | location = London | isbn =0-14-028696-9 | pages = 31–32 | quote =He [the Komsomol leader, Maltsev] went on to boast that '2 million of our children were born in Germany.' }} and the destruction of multiple major cities. The remaining national territory and Berlin were partitioned by the [[Allies of World War II|Allies]] into four military occupation zones.
The western sectors, controlled by France, the United Kingdom, and the United States, were merged on 23 May 1949, to form the ''[[West Germany|Federal Republic of Germany]]'' (''Bundesrepublik Deutschland''); on 7 October 1949, the Soviet Zone became the ''[[East Germany|German Democratic Republic]]'' (''Deutsche Demokratische Republik'', or DDR). They were, mainly outside Germany, informally known as "West Germany" and "East Germany", and the two parts of Berlin as "[[West Berlin]]" and "[[East Berlin]]". East Germany selected East Berlin as its capital, while West Germany chose [[Bonn]]. However, West Germany declared the status of its capital Bonn as provisional, in order to emphasise its stance that the two-state solution was an artificial ''status quo'' that was to be overcome one day.{{cite book | last = Wise | first = Michael Z. | title = Capital dilemma: Germany's search for a new architecture of democracy| year = 1998| publisher = Princeton Architectural Press | isbn =978-1-56898-134-5 | page = 23 | chapter = Bonn, Capital of Self-Effacement}}
West Germany, established as a federal parliamentary republic with a "[[social market economy]]", was allied with the United States, the UK and France. The country came to enjoy prolonged economic growth beginning in the early 1950s ({{lang|de|''[[Wirtschaftswunder]]''}}). West Germany joined [[NATO]] in 1955 and was a founding member of the [[European Economic Community]] in 1957. On 1 January 1957, following a plebiscite, [[Saarland]] became part of West Germany by virtue of article 23 {{lang|de|''[[Grundgesetz]]''}}, having been occupied by France since 1947.
[[File:Thefalloftheberlinwall1989.JPG|thumb|left|The [[Berlin Wall]] in front of the [[Brandenburg Gate]] shortly after its opening in 1989]]
East Germany was an [[Eastern bloc]] state under political and military control by the [[USSR]] via the latter's occupation forces and the [[Warsaw Pact]]. While claiming to be a democracy, political power was solely executed by leading members (''[[Politburo]]'') of the communist-controlled [[Socialist Unity Party of Germany|SED (''Socialist Unity Party of Germany'')]]. Their power was ensured by the [[Stasi]], a secret service of immense size, and a variety of SED suborganizations controlling every aspect of society. In return, the basic needs of the population were satisfied at low cost by the state. A Soviet-style [[command economy]] was set up; later, the GDR became a [[Comecon]] state. While [[Communist propaganda|East German propaganda]] was based on the benefits of the GDR's social programs and the alleged constant threat of a West German invasion, many of her citizens looked to the West for political freedoms and economic prosperity.{{cite news | last = Colchester | first = Nico | url = http://www.ft.com/cms/s/504285c4-68b6-11da-bd30-0000779e2340,dwp_uuid=6f876a3c-e19f-11da-bf4c-0000779e2340.html | title = D-mark day dawns | newspaper = [[Financial Times]] | location = London | date = 1 January 2001 | accessdate = 7 December 2006 }} The [[Berlin Wall]], built in 1961 to stop East Germans from escaping to West Germany, became a symbol of the [[Cold War]].
Tensions between East and West Germany were somewhat reduced in the early 1970s by Chancellor [[Willy Brandt]]'s {{lang|de|''[[Ostpolitik]]''}}, which included the ''de facto'' acceptance of Germany's territorial losses in World War II.
In summer 1989, Hungary decided (2 May) to dismantle the [[Iron Curtain]] and open the borders (23 August), causing an exodus of thousands of [[East Germans]] (11 September) to West Germany via Hungary.
This had devastating effects on the [[GDR]], with [[Monday demonstrations in East Germany|mass demonstrations]]. The East German authorities unexpectedly eased the border restrictions in November, allowing East German citizens to travel to the West. Originally intended as a pressure valve to retain East Germany as a state, the opening of the border actually led to an acceleration of the [[Die Wende|''Wende'' reform process]] in East Germany, which finally concluded with the ''[[Treaty on the Final Settlement with Respect to Germany|Two Plus Four Treaty]]'' a year later on 12 September 1990, under which the four occupying powers renounced their rights under the Instrument of Surrender, and Germany regained full [[sovereignty]]. This permitted [[German reunification]] on 3 October 1990, with the accession of the five re-established states in the former GDR ([[New states of Germany|new states]] or "neue Länder").
===Berlin Republic and the EU 1990–present===
{{Main|History of Germany since 1990}}
[[File:Euro banknotes.png|thumb| A new currency, the [[Euro]], began circulating on 1 January 2002.]]
Based on the Bonn-Berlin Act, adopted by the parliament on 10 March 1994, Berlin once again became the capital of the reunified Germany, while Bonn obtained the unique status of a ''Bundesstadt'' (federal city) retaining some federal ministries.{{de icon}} [http://www.wdr.de/themen/politik/nrw03/bonn_berlin/060914.jhtml?rubrikenstyle=politik Landtag einstimmig gegen Komplettumzug]{{dead link|date=September 2010}} WDR; 14 September 2006.{{de icon}} [http://www.wdr.de/themen/politik/deutschland/bonn_berlin/060619.jhtml?rubrikenstyle=politik Deutschlands heimliche Hauptstadt Landtag einstimmig gegen Komplettumzug]{{dead link|date=September 2010}} WDR; 20 June 2006. The relocation of the government was completed in 1999.
Since reunification, Germany has taken a more active role in the [[European Union]] and NATO. Germany sent a peacekeeping force to secure stability in the [[1999 NATO bombing in Yugoslavia|Balkans]] and sent a force of [[Bundeswehr|German troops]] to [[Afghanistan]] as part of a NATO effort to provide [[War in Afghanistan (2001–present)|security in that country]] after the ousting of the [[Taliban]].Dempsey, Judy. [http://www.iht.com/articles/2006/10/31/news/germany.php Germany is planning a Bosnia withdrawal] International Herald Tribune. 31 October 2006. Retrieved 30 November 2006. These deployments were controversial, since after the war, Germany was bound by domestic law only to deploy troops for defence roles. Deployments to foreign territories were understood not to be covered by the defence provision; however, the parliamentary vote on the issue effectively legalised the participation in a peacekeeping context.
In 2005, [[Angela Merkel]] was elected the first female Chancellor of Germany. From 2005 to 2009, she led a [[grand coalition (Germany)|grand coalition]] supported by her own parliamentary group (consisting of [[Christian Democratic Union (Germany)|CDU]] and [[Christian Social Union of Bavaria|CSU]]) and the [[Social Democratic Party of Germany|Social Democratic Party (SPD)]]. Following general elections on 27 September 2009, Merkel built the current coalition government replacing the Social Democratic Party with the [[Free Democratic Party (Germany)|Free Democratic Party (FDP)]].
==Geography==
{{Main|Geography of Germany}}
[[File:Deutschland topo.jpg|thumb|upright|right|Topographic map]]
Germany is located in the Central Europe, the territory covers {{convert|357021|km2|sqmi|0|abbr=on}}, consisting of {{convert|349223|km2|sqmi|0|abbr=on}} of land and {{convert|7798|km2|sqmi|0|abbr=on}} of water. It is the seventh largest country by area in Europe and the 63rd largest in the world. It lies mostly between latitudes [[47th parallel north|47°]] and [[55th parallel north|55° N]] (the tip of [[Sylt]] is just north of 55°), and longitudes [[5th meridian east|5°]] and [[16th meridian east|16° E]].
Elevation ranges from the mountains of the [[Alps]] (highest point: the [[Zugspitze]] at {{convert|2962|m|ft|0|disp=s}}) in the south to the shores of the [[North Sea]] (Nordsee) in the north-west and the [[Baltic Sea]] (Ostsee) in the north-east. Between lie the forested uplands of central Germany and the low-lying lands of northern Germany (lowest point: [[Wilstermarsch]] at {{convert|3.54|m|ft|1|disp=s}} below sea level), traversed by some of Europe's major rivers such as the [[Rhine]], [[Danube]] and [[Elbe]].{{Cite web| url = https://www.cia.gov/library/publications/the-world-factbook/geos/gm.html| title = CIA Factbook | accessdate = 23 October 2010| author=CIA | authorlink = Central Intelligence Agency}}
Germany shares borders with more European countries than any other country on the continent. Its neighbours are Denmark in the north, Poland and the Czech Republic in the east, Austria and Switzerland in the south, France and [[Luxembourg]] in the south-west and Belgium and the Netherlands in the north-west.
The most significant natural resources are iron ore, coal, [[potash]], timber, [[lignite]], [[uranium]], copper, natural gas, [[salt]], nickel, [[arable land]] and water.
===Administrative divisions===
{{Main|States of Germany|List of administrative divisions of Germany}}
Germany comprises [[States of Germany|16 states]] (''Bundesländer''), which are further subdivided into 439 [[Districts of Germany|districts]] (''Kreise'') and cities (''kreisfreie Städte'').
{{German Federal States|options=float:left; font-size:90%; border:3px; max-width:480px; width:50%;}}
{| style="background:none;" cellspacing="2px"
|
{| class="sortable wikitable" style="text-align:left; font-size:90%;"
|- style="font-size:100%; text-align:right;"
! style="width:140px;"| [[States of Germany|State]] !! style="width:85px;"| Capital !! style="width:85px;"| Area (km²)!! style="width:85px;"| Population
|-
| [[Baden-Württemberg]] || [[Stuttgart]] || style="text-align:right"|35,752|| style="text-align:right"|10,717,000
|-
| [[Bavaria]] || [[Munich]] || style="text-align:right"|70,549|| style="text-align:right"|12,444,000
|-
| Berlin || Berlin ||style="text-align:right"|892|| style="text-align:right"|3,400,000
|-
| [[Brandenburg]] || [[Potsdam]] || style="text-align:right"|29,477|| style="text-align:right"|2,568,000
|-
| [[Bremen (state)|Bremen]] || [[Bremen]] || style="text-align:right"|404|| style="text-align:right"|663,000
|-
| [[Hamburg]] || [[Hamburg]] ||style="text-align:right"|755|| style="text-align:right"|1,735,000
|-
| [[Hesse]] || [[Wiesbaden]] || style="text-align:right"|21,115|| style="text-align:right"|6,098,000
|-
| [[Mecklenburg-Vorpommern]] || [[Schwerin]] || style="text-align:right"|23,174|| style="text-align:right"|1,720,000
|-
| [[Lower Saxony]] || [[Hanover]] || style="text-align:right"|47,618|| style="text-align:right"|8,001,000
|-
| [[North Rhine-Westphalia]] || [[Düsseldorf]] || style="text-align:right"|34,043|| style="text-align:right"|18,075,000
|-
| [[Rhineland-Palatinate]] || [[Mainz]] || style="text-align:right"|19,847|| style="text-align:right"|4,061,000
|-
| [[Saarland]] || [[Saarbrücken]] || style="text-align:right"|2,569|| style="text-align:right"|1,056,000
|-
| [[Saxony]] || [[Dresden]] || style="text-align:right"|18,416|| style="text-align:right"|4,296,000
|-
| [[Saxony-Anhalt]] || [[Magdeburg]] || style="text-align:right"|20,445|| style="text-align:right"|2,494,000
|-
| [[Schleswig-Holstein]] || [[Kiel]] || style="text-align:right"|15,763|| style="text-align:right"|2,829,000
|-
| [[Thuringia]] || [[Erfurt]] || style="text-align:right"|16,172|| style="text-align:right"|2,355,000
|}
|}
===Climate===
{{multiple image
| align = right
| image1 = Sylt Rotes Kliff.jpg
| width1 = 177
| caption1 = Summer coastal climate on the island [[Sylt]] in [[Schleswig-Holstein]]
| alt1 = Coastal dunes
| image2 = Thüringerwald.JPG
| width2 = 210
| caption2 = [[Thuringian Forest]]s during wintertime
| alt2 = Thuringian Forest
}}
Most of Germany has a [[temperate|temperate seasonal climate]] in which humid westerly winds predominate. The climate is moderated by the [[North Atlantic Current|North Atlantic Drift]], which is the northern extension of the [[Gulf Stream]]. This warmer water affects the areas bordering the North Sea including the area along the Rhine, which flows into the North Sea. Consequently in the north-west and the north, the climate is [[oceanic climate|oceanic]]; [[Precipitation (meteorology)|rainfall]] occurs year round with a maximum during summer.
Winters are mild and summers tend to be cool, though temperatures can exceed {{convert|30|°C|°F|lk=on}} for prolonged periods. In the east, the climate is more [[continental climate|continental]]; winters can be very cold, summers can be very warm, and long dry periods are often recorded. Central and southern Germany are transition regions which vary from moderately oceanic to continental.
In addition to the maritime and continental climates that predominate over most of the country, the Alpine regions in the extreme south and, to a lesser degree, some areas of the Central German Uplands have a so-called mountain climate. This climate is characterised by lower temperatures because of higher altitudes and greater precipitation caused by air becoming moisture-laden as it lifts over higher terrain.[http://www.germanculture.com.ua/library/facts/bl_climate.htm Climate In Germany] GermanCulture. Retrieved 2010-26-08.
===Biodiversity===
{{multiple image
| align = right
| width2 = 202
| image2 = Hintersee.jpg
| caption2 = [[Alps|Alpine]] terrain in [[Bavaria]]. One third of the country's area is covered with woods.
| alt2 = Alpine terrain
| width1 = 180
| image1 = Cervus elaphus 11.jpg
| caption1 = [[Red Deer|Deer]] are widespread species to be found in the wild.
| alt1 = Red Deer
}}
[[Phytogeography|Phytogeographically]], Germany is shared between the Atlantic European and Central European provinces of the [[Circumboreal Region]] within the [[Boreal Kingdom]]. The territory of Germany can be subdivided into two [[ecoregion]]s: European-Mediterranean montane mixed forests and Northeast-Atlantic shelf marine.[http://www.panda.org/about_our_earth/ecoregions/ecoregion_list/ List of Ecoregions: Terrestrial Ecoregions] WWF. Retrieved 21 November 2000. The majority of Germany is covered by either [[arable land]] (33%) or [[forestry]] and [[woodland]] (31%). Only 15% is covered by permanent pastures.
Plants and animals are those generally common to middle Europe. Beeches, oaks, and other deciduous trees constitute one-third of the forests; conifers are increasing as a result of reforestation. Spruce and fir trees predominate in the upper mountains, while pine and larch are found in sandy soil. There are many species of ferns, flowers, fungi, and mosses. Fish abound in the rivers and the North Sea. Wild animals include deer, wild boar, mouflon, fox, badger, hare, and small numbers of beavers. Various migratory birds cross Germany in the spring and autumn.
The [[List of national parks of Germany|national parks in Germany]] include the [[Wadden Sea National Parks]], the [[Jasmund National Park]], the [[Vorpommern Lagoon Area National Park]], the [[Müritz National Park]], the [[Lower Oder Valley National Park]], the [[Harz National Park]], the [[Saxon Switzerland National Park]] and the [[Bavarian Forest National Park]].
Germany is known for its many [[zoological garden]]s, wildlife parks, aquaria, and bird parks.[http://www.eupedia.com/europe/zoological_gardens_europe.shtml List of famous Zoological gardens in European countries] www.eupedia.com. Retrieved 17 October 2008. More than 400 registered zoos and animal parks operate in Germany, which is believed to be the largest number in any single country of the world.[http://www.americanzoos.info/Zoofacts.html Some interesting zoo facts] www.americanzoos.info/. Retrieved 17 October 2008. The [[Berlin Zoological Garden|Zoologischer Garten Berlin]] is the oldest zoo in Germany and presents the most comprehensive collection of species in the world.{{de icon}}[http://www.zoo-berlin.de/en/understand/zahlen-fakten.html Tierstatistik 2008]{{dead link|date=September 2010}}, Zoo Berlin. Retrieved 19 November 2009.
===Environment===
{{multiple image
| align = right
| width2 = 200
| image2 = Aquila chrysaetos Flickr.jpg
| caption2 = The [[golden eagle|eagle]] is a protected bird of prey and the national [[heraldic animal]].
| alt2 = Eagle
| width1 = 185
| image1 = HeideblueteTiefental.JPG
| caption1 = The [[Lüneburg Heath Nature Reserve]] in [[Lower Saxony]]
| alt1 = Lüneburg Heath
}}
Germany is known for its environmental consciousness.[http://docs.google.com/viewer?a=v&q=cache:ULS2JqlpDgEJ:dzforum.de/downloads/020101311.pdf+Starting+points+and+focus+of+environmental+communication+site:dzforum.de&hl=en&pid=bl&srcid=ADGEESiPmGynU-6SalJJqHpsYQsnf6um5_nPW6Wz6R3HgEOQZEV2kxltI0OWaCuWNrWugnjPki98ww1R0r_-VfGBYjxnfLY2Zg8tzzwsagtvTi-w1-FgqsPK-iEYJgRjRyvzIsVQJgrF&sig=AHIEtbTTWzlVX6stOR-oJP4FxK20wmyGmA Starting points and focus of environmental communication], German Federal Environmental Foundation, 16 October 2004. Most Germans consider [[anthropogenic]] causes to be a significant factor in [[global warming]].{{Cite web| url = http://www.worldpublicopinion.org/pipa/pdf/sep07/BBCClimate_Sep07_rpt.pdf| title = All Countries Need to Take Major Steps on Climate Change: Global Poll| accessdate = 11 February 2008 | author=PIPA| date =24 September 2007| format = PDF|publisher=BBC World service Poll, carried out by Globescan| publisher=[[Program on International Policy Attitudes]] at the [[University of Maryland, College Park]]|quote=Eighty-seven percent of German respondents agree that human activity is a significant cause of climate change and 95 percent think it is necessary to take measures to deal with the problem.}} The state is committed to the [[Kyoto protocol]] and several other treaties promoting biodiversity, low emission standards, recycling, and the use of renewable energy, and supports sustainable development at a global level.[http://www.umweltbundesamt.de/uba-info-presse-e/2004/pe04-056.htm Reinforcing Germany's environmental protection industry]{{dead link|date=September 2010}} Umweltbundesamt. Retrieved 25 November 2007.
The German government has initiated wide-ranging emission reduction activities and the country's overall emissions are falling.[http://timesofindia.indiatimes.com/Earth/Germany_greenest_country_in_the_world/articleshow/3151766.cms Germany greenest country in the world] ''Times of India''. Retrieved 25 November 2008. For example, since 1964, air pollution in Germany has been regulated by strict "[[TA Luft]]" legislation. Nevertheless Germany's [[List of countries by carbon dioxide emissions per capita|carbon dioxide emissions per capita]] are among the highest in the EU, although they are significantly lower than those of Australia, Canada, Saudi Arabia and the United States.
Emissions from coal-burning utilities and industries contribute to air pollution. Acid rain, resulting from sulphur dioxide emissions, continues to damage German forests. Pollution in the Baltic Sea from raw sewage and industrial effluents from rivers in former East Germany have been reduced. The government under Chancellor Schröder announced the intention to end the use of nuclear power for producing electricity. Germany is working to meet EU commitments to identify nature preservation areas in line with the EU's Flora, Fauna, and Habitat directive. Germany's last glaciers in the Alpine region are experiencing deglaciation. Natural hazards are river flooding in spring and stormy winds occurring in all regions.
==Politics==
{{Main|Politics of Germany}}
[[File:Politisches System des Bundes und der Länder in Deutschland.svg|thumb|230px|The political system of Germany is written down in the [[Basic Law for the Federal Republic of Germany|Basic Law (Grundgesetz)]]. ]]
Germany is a [[federal republic|federal]], [[parliamentary democratic|parliamentary]], [[representative democracy|representative democratic]] republic. The German political system operates under a framework laid out in the 1949 constitutional document known as the [[Basic Law for the Federal Republic of Germany|''Grundgesetz'']] (Basic Law). By calling the document ''Grundgesetz'', rather than ''Verfassung'' (constitution), the authors expressed the intention that it would be replaced by a proper constitution once Germany was reunited as one state.
Amendments to the ''Grundgesetz'' generally require a two-thirds majority of both chambers of the parliament; the fundamental principles of the constitution, as expressed in the articles guaranteeing human dignity, the separation of powers, the federal structure, and the rule of law are valid in perpetuity.{{Cite web|url=http://www.gesetze-im-internet.de/gg/art_79.html|title=Article 79 of the ''Grundgesetz''|work=Bundesministerium der Justiz|publisher=www.gesetze-im-internet.de|language=German|accessdate=7 December 2008}} Despite the initial intention, the ''Grundgesetz'' remained in effect after the [[German reunification]] in 1990, with only minor amendments.
===Government===
[[File:Christian Wulff 2008-04-17.jpg|thumb|left|[[Christian Wulff]] has been the [[President of Germany]] since June 2010. The head of state holds primarily representative powers.]]
The President, [[Christian Wulff]], is the [[head of state]] and invested primarily with representative responsibilities and powers. He is elected by the ''[[Bundesversammlung (Germany)|Bundesversammlung]]'' (federal convention), an institution consisting of the members of the ''Bundestag'' and an equal number of state delegates.
The second highest official in the [[German order of precedence]] is the ''Bundestagspräsident'' ([[President of the Bundestag|President of the ''Bundestag'']]), who is elected by the ''Bundestag'' and responsible for overseeing the daily sessions of the body.
The third-highest official and the [[head of government]] is the Chancellor, who is appointed by the ''Bundespräsident'' after being elected by the ''Bundestag''. The Chancellor can be removed by a [[constructive vote of no confidence]] by the ''Bundestag'', where constructive implies that the ''Bundestag'' simultaneously has to elect a successor.
[[File:Reichstag building Berlin view from west before sunset.jpg|thumb|290px|The [[Reichstag (building)|Reichstag]] in Berlin is the site of the German parliament (Bundestag)]]
The [[Chancellor of Germany|Chancellor]]—currently [[Angela Merkel]]—is the [[head of government]] and exercises [[executive (government)|executive power]], similar to the role of a Prime Minister in other [[parliamentary system|parliamentary democracies]]. Federal [[legislative power]] is vested in the parliament consisting of the ''[[Bundestag]]'' (Federal Diet) and ''[[Bundesrat of Germany|Bundesrat]]'' (Federal Council), which together form a unique type of legislative body. The ''Bundestag'' is elected through [[direct election]]s, by [[proportional representation]] ([[mixed member proportional representation|mixed-member]]). The members of the ''Bundesrat'' represent the governments of the [[States of Germany|sixteen federated states]] and are members of the state cabinets. The respective state governments have the right to appoint and remove their envoys at any time.
Since 1949, the party system has been dominated by the [[Christian Democratic Union (Germany)|Christian Democratic Union]] and the [[Social Democratic Party of Germany]] with all chancellors hitherto being member of either party. However, the smaller liberal [[Free Democratic Party (Germany)|Free Democratic Party]] (which has had members in the ''Bundestag'' since 1949) and the [[Alliance '90/The Greens]] (which has controlled seats in parliament since 1983) have also played important roles,[http://countrystudies.us/germany/159.htm Christian Democratic Union/Christian Social Union] U.S. Library of Congress. Retrieved 7 December 2006. as they are regularly the smaller partner of a [[coalition government]].
===Law===
{{Main|Judiciary of Germany|Law enforcement in Germany}}
[[File:Karlsruhe bundesverfassungsgericht.jpg|right|thumb|The [[Federal Constitutional Court of Germany|Federal Constitutional Court]] (Bundesverfassungsgericht) in [[Karlsruhe]]]]
The [[Judiciary of Germany]] is independent of the executive and the legislative branches. Germany has a [[civil law (legal system)|civil or statute law system]] that is based on [[Roman law]] with some references to [[Germanic law]]. The ''[[Federal Constitutional Court of Germany|Bundesverfassungsgericht]]'' (Federal Constitutional Court), located in Karlsruhe, is the German Supreme Court responsible for constitutional matters, with power of [[judicial review]].[http://www.bundesverfassungsgericht.de/en/index.html Federal Constitutional Court], Bundesverfassungsgericht.de. Retrieved 13 April 2007. It acts as the highest legal authority and ensures that legislative and judicial practice conforms to the [[Basic Law for the Federal Republic of Germany]] (Basic Law). It acts independently of the other state bodies, but cannot act on its own behalf.
Germany's supreme court system, called ''Oberste Gerichtshöfe des Bundes'', is specialised. For civil and criminal cases, the highest court of appeal is the [[Federal Court of Justice of Germany|Federal Court of Justice]], located in [[Karlsruhe]] and [[Leipzig]]. The courtroom style is [[inquisitorial system|inquisitorial]]. Other Federal Courts are the [[Federal Labor Court of Germany|Federal Labour Court]] in [[Erfurt]], the [[Bundessozialgericht|Federal Social Court]] in [[Kassel]], the [[Federal Finance Court of Germany|Federal Finance Court]] in [[Munich]] and the [[Federal Administrative Court of Germany|Federal Administrative Court]] in Leipzig.
[[File:HH Polizeihauptmeister MZ.jpg|thumb|150px|left|German [[Landespolizei|state police]] officer in [[Hamburg]]]]
[[Criminal law]] and [[private law]] are codified on the national level in the ''[[Strafgesetzbuch]]'' and the ''[[Bürgerliches Gesetzbuch]]'' respectively. The German penal system is aimed towards rehabilitation of the criminal; its secondary goal is the protection of the general public.{{de icon}} [http://www.gesetze-im-internet.de/stvollzg/__2.html § 2 Strafvollzugsgesetz] Bundesministerium der Justiz. Retrieved 19 November 2009. To achieve the latter, a convicted criminal can be put in preventive detention (''Sicherungsverwahrung'') in addition to the regular sentence if he is considered to be a threat to the general public. The ''[[Völkerstrafgesetzbuch]]'' regulates the consequences of [[crimes against humanity]], [[genocide]] and [[war crimes]]. It gives German courts [[universal jurisdiction]] if prosecution by a court of the country where the crime was committed, or by an international court, is not possible.
Legislative power is divided between the federation and the state level. The Basic Law presumes that all legislative power remains at the state level unless otherwise designated by the Basic Law itself.
Any federal law overrides state law if the legislative power lies at the federal level. A famous example is the purported Hessian provision for the death penalty, which goes against the ban on capital punishment under the Basic Law, rendering the Hessian provision invalid. The ''[[Bundesrat of Germany|Bundesrat]]'' is the federal organ through which the states participate in national legislation. State participation in federal legislation is necessary if the law falls within the area of concurrent legislative power, requires states to administer federal regulations, or is so designated by the Basic Law. Every state has its own constitutional court. The ''[[Amtsgericht]]e'', ''Landgerichte'' and ''Oberlandesgerichte'' are state courts of [[general jurisdiction]]. They are competent whether the action is based on federal or state law.
Many of the fundamental matters of [[administrative law]] remain in the jurisdiction of the states, though most states base their own laws in that area on the 1976 ''Verwaltungsverfahrensgesetz'' (Administrative Proceedings Act) covering important points of administrative law. The ''Oberverwaltungsgerichte'' are the highest level of administrative jurisdiction concerning the state administrations, unless the question of law concerns federal law or state law identical to federal law. In such cases, final appeal to the Federal Administrative Court is possible.
===Foreign relations===
{{Main|Foreign relations of Germany}}
[[File:33rdG8Leaders.jpg|thumb|Chancellor [[Angela Merkel]], the head of government, hosting the [[G8]] summit in [[Heiligendamm]].]]
Germany maintains a network of 229 diplomatic missions abroad and holds relations with more than 190 countries.[http://www.auswaertiges-amt.de/EN/AAmt/Auslandsvertretungen/Uebersicht_node.html German Missions Abroad] German Federal Foreign Office. Retrieved 31 December 2010. As member it is the largest contributor to the budget of the European Union (providing 20%) and third largest contributor to the United Nations (providing 8%). Germany is a member of the NATO defence alliance, the Organisation of Economic Co-operation and Development (OECD), the G8, the G20, the World Bank and the International Monetary Fund (IMF).
Germany has played a leading role in the European Union since its inception and has maintained a [[Franco-German cooperation|strong alliance with France]] since the end of World War II. The alliance was especially close in the late 1980s and early 1990s under the leadership of [[Christian Democratic Union (Germany)|Christian Democrat]] [[Helmut Kohl]] and [[Socialist Party (France)|Socialist]] [[François Mitterrand]]. Germany is at the forefront of European states seeking to advance the creation of a more unified European political, defence, and security apparatus.[http://www.elysee.fr/elysee/anglais/speeches_and_documents/2004/declaration_by_the_franco-german_defence_and_security_council.1096.html Declaration by the Franco-German Defence and Security Council] Elysee.fr 13 May 3004. Retrieved 3 December 2006. For a number of decades after WWII, the Federal Republic of Germany kept a notably low profile in international relations, because of both its recent history and its occupation by foreign powers.Glaab, Manuela. [http://web.archive.org/web/20071121072523/http://en.internationalepolitik.de/archiv/2003/spring2003/german-foreign-policy.html German Foreign Policy: Book Review] Internationale Politik. Spring 2003. Retrieved 3 January 2007.
[[File:Rometreaty.jpg|thumb|left|Germany is a founding member of the [[European Economic Community|EEC]] in 1957, which became the European Union in 1993. It maintains close relations with its neighbours to coordinate EU politics.]]
During the Cold War, Germany's partition by the [[Iron Curtain]] made it a symbol of East-West tensions and a political battleground in Europe. However, Willy Brandt's Ostpolitik was a key factor in the ''[[détente]]'' of the 1970s.Harrison, Hope. {{PDFlink|[http://www.ghi-dc.org/files/publications/bu_supp/supp1/supp-01_005.pdf ''The Berlin Wall, Ostpolitik and Détente]|91.1 KB}} German historical institute, Washington, DC, Bulletin supplement 1, 2004, ''American détente and German ostpolitik, 1969–1972". In 1999, Chancellor [[Gerhard Schröder]]'s government defined a new basis for German foreign policy by taking a full part in the decisions surrounding the [[Kosovo War|NATO war against Yugoslavia]] and by sending German troops into combat for the first time since World War II.[http://www.dw-world.de/dw/article/0,2144,1741310,00.html Germany's New Face Abroad] [[Deutsche Welle]]. 14 October 2005. Retrieved 3 December 2006.
[[File:Defense ministers of NATO 2000.jpg|thumb|right|West Germany became a [[NATO]] member in 1955. (Defence ministers in 2000)]]
The governments of Germany and the United States are close political allies.[http://www.state.gov/r/pa/ei/bgn/3997.htm Background Note: Germany] U.S. Department of State. 6 July 2006. Retrieved 3 December 2006. The 1948 [[Marshall Plan]] and strong cultural ties have crafted a strong bond between the two countries, although Schröder's very vocal opposition to the [[Iraq War]] suggested the end of [[Atlanticism]] and a relative cooling of German-American relations.[http://www.economist.com/world/europe/displaystory.cfm?story_id=7141311 ''Ready for a Bush hug?''], [[The Economist]], 6 July 2006. Retrieved 31 December 2006. The two countries are also economically interdependent: 8.8% of German exports are U.S.-bound and 6.6% of German imports originate from the U.S.{{PDFlink|[http://germany.usembassy.gov/germany/img/assets/9336/econ_factsheet_may2006.pdf U.S.-German Economic Relations Factsheet]|32.8 KB}} U.S. Embassy in Berlin. May 2006. Retrieved 3 December 2006. Other signs of the close ties include the continuing position of German-Americans as the largest ethnic group in the U.S.[http://www.census.gov/Press-Release/www/releases/archives/census_2000/001870.html German Still Most Frequently Reported Ancestry] [[U.S. Census Bureau]] 30 June 2004. Retrieved 3 December 2006. {{Dead link|date=June 2010| bot=DASHBot}} and the status of [[Ramstein Air Base]] (near [[Kaiserslautern]]) as the largest U.S. military community outside the U.S.[http://benefits.military.com/misc/installations/Base_Content.jsp?id=1675 Kaiserslautern, Germany Overview] U.S. Military. Retrieved 3 December 2006.
The development policy of the Federal Republic of Germany is an independent area of German foreign policy. It is formulated by the Federal Ministry for Economic Cooperation and Development (BMZ) and carried out by the implementing organisations. The German government sees development policy as a joint responsibility of the international community.[http://www.bmz.de/en/index.html Aims of German development policy] Federal Ministry for Economic Cooperation and Development 10 April 2008. Retrieved 7 December 2008. It is the world's third biggest aid donor after the United States and France.[http://www.oecd.org/dataoecd/17/9/44981892.pdf Table: Net Official Development Assistance 2009] OECD Germany spent 0.37 per cent of its gross domestic product (GDP) on development, which is below the government's target of increasing aid to 0.51 per cent of GDP by 2010. The international target of 0.7% of GNP would have not been reached either.
===Military===
{{Main|Bundeswehr}}
[[File:Fregatte Mecklenburg-Vorpommern F218.jpg|right|thumb|The ''Mecklenburg-Vorpommern'' participated in a [[United Nations Interim Force in Lebanon|UNIFIL II]] operation off the coast of [[Lebanon]].]]
Germany's military, the ''[[Bundeswehr]]'', is a military force with ''[[German Army|Heer]]'' (Army), ''[[German Navy|Marine]]'' (Navy), ''[[Luftwaffe]]'' (Air Force), [[Central Medical Services|''Zentraler Sanitätsdienst'']] (Central Medical Services) and ''[[Streitkräftebasis]]'' (Joint Support Service) branches. Military service is compulsory for men at the age of 18, and conscripts serve six-month tours of duty. Conscientious objectors may instead opt for an equal length of ''[[Zivildienst]]'' (roughly translated as civilian service), or a six year commitment to (voluntary) emergency services like a [[Volunteer fire department|fire department]], the [[German Red Cross|Red Cross]] or the [[Technisches Hilfswerk|THW]]. In 2005, military spending was an estimated 1.5% of the country's GDP. In peacetime, the Bundeswehr is commanded by the Minister of Defence. If Germany went to war, which according to the constitution is allowed only for defensive purposes, the Chancellor would become commander in chief of the ''Bundeswehr''.{{de icon}} [http://www.bundestag.de/parlament/funktion/gesetze/grundgesetz/gg_10a.html Grundgesetz für die Bundesrepublik Deutschland: Article 115a]{{dead link|date=September 2010}} Bundestag.de. Retrieved 19 November 2009. {{Wayback|url=http://www.bundestag.de/parlament/funktion/gesetze/grundgesetz/gg_10a.html|date =20080614163824|bot=DASHBot}}{{dead link|date=September 2010}}
[[File:Eurofighter 9803.ogg|thumb|thumbtime=32|left|A [[Eurofighter Typhoon|Eurofighter]] flight demonstration]]
The Bundeswehr employs 200,500 professional soldiers, 55,000 18–25 year-old conscripts who serve for at least six months under current rules, and 2,500 active reservists at any given time. Roughly 300,000 reservists are available to the Armed Forces and participate in defence exercises as well as deployments abroad.
Since 2001 women can serve in all functions of service without restriction, but they are not subject to conscription. There are presently around 14,500 women on active duty and a number of female reservists who take part in all duties including peacekeeping missions and other operations. Two female medical officers have been promoted to a General rank so far.
{{As of|2009|11}}, the German military had about 8,300 troops stationed in foreign countries as part of various international peacekeeping forces, including 2,470 Bundeswehr soldiers in [[Kosovo]], 4,520 German troops in the NATO-led [[International Security Assistance Force|ISAF]] force in [[Afghanistan]] and [[Uzbekistan]], and 450 troops with [[UNIFIL]] in [[Lebanon]].{{Cite web|url=http://www.bundeswehr.de/portal/a/bwde/einsaetze/einsatzzahlen?yw_contentURL=/C1256EF4002AED30/W264VFT2439INFODE/content.jsp|title=Einsatzzahlen – Die Stärke der deutschen Einsatzkontingente|accessdate=8 November 2009|publisher=[[Bundeswehr]]|language=German}}
{{clearleft}}
==Economy==
{{Main|Economy of Germany}}
[[File:S-Klasse W221.jpg|thumb|A Mercedes-Benz automobile. Germany was the world's leading exporter of goods from 2003 to 2008.{{Cite news| url=http://www.nytimes.com/2010/02/20/business/economy/20charts.html | work=The New York Times | title=A Shift in the Export Powerhouses | first=Floyd | last=Norris | date=20 February 2010 | accessdate=31 March 2010}}]]
Germany has a [[social market economy]] characterised by a highly qualified [[labour force]], a developed [[infrastructure]], a large [[capital stock]], a low level of corruption,{{Cite web |url=http://www.transparency.org/policy_research/surveys_indices/cpi/2009/cpi_2009_table |title=CPI 2009 table |publisher=Transparency International |accessdate=18 November 2009}} and a high level of [[innovation]].{{Cite web|url=http://www.nam.org/innovationreport.pdf |title=The Innovation Imperative in Manufacturing: How the United States Can Restore Its Edge |format=PDF |date= |accessdate=26 September 2010}} It has the largest national economy in Europe, the [[List of countries by GDP (nominal)|fourth largest by nominal GDP]] in the world, and ranked [[List of countries by GDP (PPP)|fifth by GDP (PPP)]] in 2009. The [[Tertiary sector of industry|service sector]] contributes around 70% of the total GDP, industry 29.1%, and agriculture 0.9%. In July 2010 the average national unemployment rate was 7.5%, ranging from 4.1% in Bavaria to 13.6% in Berlin. First estimations indicate a 3.6% increase in the price-adjusted Gross Domestic Product (GDP) for 2010, which followed a 4.7% drop recorded in 2009."German Economy Experiences Record Growth in 2010" [http://www.germany.info/Vertretung/usa/en/__pr/P__Wash/2011/01/12__GDP__2010__PR.html German Embassy Press Release 12 Jan. 2011]
[[File:Eurozone.svg|thumb|upright|left|Germany is part of a monetary union, the [[eurozone]] (dark blue), and of the [[Internal Market (European Union)|EU single market]]. ]]
Since the age of [[industrialisation]], the country has been a driver, innovator, and beneficiary of an ever more globalised economy. Germany is a founding member of the EU, the G8 and the G20 and was the world's largest exporter from 2003 to 2008. In 2009 it remains the [[List of countries by exports|second largest exporter]] and third largest importer of goods while generating a [[trade surplus]] of $189.7 billion.[Germany Trade Statistics 2009], [[suite101.com]], 1 April 2010. Retrieved 25 August 2010.
Most of the country's export products are in engineering, especially in machinery, automobiles, chemical goods and metals. Germany is the leading producer of wind turbines and solar power technology in the world. The largest annual international trade fairs and congresses are held in several German cities such as Hanover, Frankfurt, and Berlin.[http://web.archive.org/web/20061210163253/http://www.german-renewable-energy.com/Renewables/Navigation/Englisch/wind-power.html Wind Power] Federal Ministry of Economics and Technology (Germany) Retrieved 30 November 2006.
Germany is an advocate of closer European economic and political integration. Its commercial policies are increasingly determined by agreements among European Union (EU) members and EU [[single market]] legislation. Germany introduced the common European currency, the euro on 1 January 2002. Its monetary policy is set by the [[European Central Bank]] in Frankfurt.
Two decades after [[German reunification]], [[standard of living|standards of living]] and per capita incomes remain significantly higher in the states of the former West Germany than in the former East.Berg, S., Winter, S., Wassermann, A. [http://www.spiegel.de/international/spiegel/0,1518,373639,00.html The Price of a Failed Reunification] Spiegel Online International. 5 September 2005. Retrieved 28 November 2006. The modernisation and integration of the eastern German economy continues to be a long-term process scheduled to last until the year 2019, with annual transfers from west to east amounting to roughly $80 billion.{{Cite news| url=http://www.nytimes.com/2009/06/19/world/europe/19germany.html | work=The New York Times | title=In East Germany, a Decline as Stark as a Wall | first=Nicholas | last=Kulish | date=19 June 2009 | accessdate=31 March 2010}}
The nominal GDP of Germany contracted in the second and third quarters of 2008, putting the country in a technical recession following a global and [[Late 2000s recession in Europe|European recession]] cycle.{{Cite news| url= http://www.guardian.co.uk/world/2008/nov/14/oecd-recession-germany-inflation-deflation | title= Germany officially in recession as OECD expects US to lead recovery|work=The Guardian |location=UK |date=14 November 2008 | first=Kathryn | last=Hopkins | accessdate=31 March 2010}} In January 2009 the German government under Angela Merkel approved a €50 billion ($70 billion) economic stimulus plan to protect several sectors from a downturn and a subsequent rise in unemployment rates.{{Cite news| url= http://www.france24.com/en/20090106-germany-agrees-new-50-billion-euro-stimulus-plan| title= Germany agrees on 50-billion-euro stimulus plan|publisher=France 24| date=6 January 2009}}
===Corporations===
[[File:Siemens Palais.JPG|thumb|Seat of the [[Siemens|Siemens AG]] in Munich. The city is the headquarter of 7 [[Fortune Global 500]] companies.]]
[[File:Frankfurt Am Main-Stadtansicht von der Deutschherrnbruecke zu Beginn der Abenddaemmerung.jpg|thumb|[[Frankfurt am Main|Frankfurt]] is a financial [[global city]], seat of the [[European Central Bank]], and an international aviation hub.]]
Of the world's 500 largest stock market listed companies measured by [[revenue]] in 2010, the [[Fortune Global 500]], 37 are headquartered in Germany. 30 Germany based companies are included in the [[DAX]], the German stock market index, trading on the [[Frankfurt Stock Exchange]].
Well known global brands are [[Mercedes-Benz]], [[BMW]], [[Adidas]], [[Audi]], [[Porsche]], [[Volkswagen]], [[DHL Express|DHL]], [[T-Mobile]], [[Lufthansa]], [[SAP AG|SAP]], and [[Nivea]].[http://web.archive.org/web/20080106165521/http://bwnt.businessweek.com/brand/2006/ The 100 Top Brands 2006], Businessweek. Retrieved 26 November 2007.
Germany is recognised for its highly specialised [[small and medium enterprises]]. Around 1000 of these mostly family-owned companies are global market leaders in their segment and are usually labelled [[Hidden Champions|hidden champions]].[http://www.businessweek.com/news/2010-09-23/germany-has-1-000-market-leading-companies-manager-magazin-says.html Germany Has 1,000 Market-Leading Companies, Manager-Magazin Says], Businessweek. Retrieved 23 September 2010.
The list includes the largest companies by turnover in 2009. Unranked are the largest bank and the largest insurance company in 2007:
{| class="wikitable sortable" style="text-align:right;"
|- style="background:#efefef;"
! style="text-align:center;"| Rank[http://money.cnn.com/magazines/fortune/global500/2010/countries/Germany.html], CNN Money. Retrieved 7 August 2010.|| style="text-align:center;"| Name|| style="text-align:center;"| Headquarters|| style="text-align:center;"| Revenue
(Mil. €) || style="text-align:center;"| Profit
(Mil. €) || style="text-align:center;"| Employees
(World)
|-
| 1. || style="text-align:left;"| [[Volkswagen AG]] || style="text-align:left;"| [[Wolfsburg]] || 108,897 || 4,120 || 329,305
|-
| 2. || style="text-align:left;"| [[Daimler AG]] || style="text-align:left;"| [[Stuttgart]] || 99,399 || 3,985 || 272,382
|-
| 3. || style="text-align:left;"| [[Siemens AG]] || style="text-align:left;"| [[Munich]]/Berlin || 72,488 || 3,806 || 398,200
|-
| 4. || style="text-align:left;"| [[E.ON|E.ON AG]] || style="text-align:left;"| [[Düsseldorf]] || 68,731 || 7,204 || 87,815
|-
| 5. || style="text-align:left;"| [[Metro AG]] || style="text-align:left;"| [[Düsseldorf]] || 64,337 || 825 || 242,378
|-
| 6. || style="text-align:left;"| [[Deutsche Post AG]] || style="text-align:left;"| [[Bonn]] || 63,512 || 1,389 || 475,100
|-
| 7. || style="text-align:left;"| [[Deutsche Telekom AG]] || style="text-align:left;"| [[Bonn]] || 62,516 || 569 || 241,426
|-
| 8. || style="text-align:left;"| [[BASF|BASF SE]] || style="text-align:left;"| [[Ludwigshafen am Rhein|Ludwigshafen]] || 57,951 || 4,065 || 95,175
|-
| 9. || style="text-align:left;"| [[BMW AG]] || style="text-align:left;"| [[Munich]] || 56,018 || 3,126 || 107,539
|-
| 10. || style="text-align:left;"| [[ThyssenKrupp AG]] || style="text-align:left;"| [[Essen]]/[[Duisburg]] || 51,723 || 2,102 || 191,350
|}
===Transport===
{{Main|Transport in Germany|Rail transport in Germany}}
[[File:ICE 3M KRM.jpg|thumb|The [[ICE 3]] train]]
With its central position in Europe, Germany is an important transportation hub. This is reflected in its dense and modern transportation networks. The extensive motorway ([[Autobahn]]) network ranks worldwide third largest in its total length and features a lack of blanket speed limits on the majority of routes.{{Cite web|authorlink=www.adac.de/verkehrs-experten|title=Autobahn-Temporegelung|publisher=[[ADAC]]|date=October 2007|url=http://www.adac.de/images/SP_Tempolimit_V%201%2E0_tcm8-199598.pdf|format=PDF|language=German|accessdate=15 November 2008}} {{Dead link|date=June 2010| bot=DASHBot}}
Germany has established a polycentric network of [[high-speed rail|high-speed trains]]. The [[InterCityExpress]] or ''ICE'' is the most advanced service category of the [[Deutsche Bahn]] and serves major German cities as well as destinations in neighbouring countries. The train maximum speed varies between 160 km/h and 300 km/h. Connections are offered at either 30-minute, hourly, or two-hourly intervals.{{de icon}} [http://web.archive.org/web/20070809140315/http://www.db.de/site/bahn/de/unternehmen/investor__relations/finanzberichte/geschaeftsbericht/geschaeftsbericht__2006.html Geschäftsbericht 2006 der Deutschen Bahn AG], [[Deutsche Bahn]]. Retrieved 19 November 2009.
The largest German airports are the [[Frankfurt International Airport]] and the [[Munich International Airport]], both are global hubs of [[Lufthansa]]. Other major airports are [[Tegel Airport|Berlin Tegel]], [[Schönefeld Airport|Berlin Schönefeld]], [[Düsseldorf International Airport|Düsseldorf]], [[Hamburg Airport|Hamburg]], [[Cologne Bonn Airport|Cologne-Bonn]], [[Leipzig/Halle Airport|Leipzig/Halle]] and in the future [[Berlin Brandenburg International Airport]].
===Energy===
{{Main|Energy in Germany}}
[[File:Windgermany.JPG|thumb|The largest [[solar power]] and [[Wind power in the European Union|second-largest wind power]] capacity in the world is installed in Germany.]]
Germany is the world's fifth largest consumer of energy, and two-thirds of its primary energy was imported in 2002. In the same year, Germany was Europe's largest consumer of electricity, totalling 512.9 terawatt-hours. Government policy promotes energy conservation and the development of [[renewable energy]] sources, such as solar, [[wind power|wind]], [[biomass]], hydroelectric, and [[geothermal power|geothermal]] energy. As a result of energy-saving measures, [[Efficient energy use|energy efficiency]] has been improving since the beginning of the 1970s. The government has set the goal of meeting half the country's energy demands from renewable sources by 2050.
In 2000, the government and the [[Nuclear power in Germany|German nuclear power industry]] agreed to phase out all [[nuclear power plant]]s by 2021.[http://news.bbc.co.uk/2/hi/europe/4295389.stm Germany split over green energy], BBC. Retrieved 13 April 2007. Renewable energy still plays a more modest role in energy consumption. In 2006, energy consumption was met by the following sources: oil (36%); coal, including lignite (24%); natural gas (23%); nuclear (13%); hydro-electric and wind power (1%); and other (4%). However, the share of renewable energy in electricity supply has been rapidly increasing, reaching 14% in 2007. The German government has set a new target to increase this share to 27% by 2020.
===Technology===
{{Main|German inventors and discoverers}}
[[File:Fächertextur.jpg|thumb|left|[[Liquid crystal]] visualised by a polarising microscope. Germany is a pioneer research center for [[nanotechnology]] and [[material engineering]].[http://www.new-delhi.diplo.de/Vertretung/newdelhi/en/06/__HB__Science__Technology.html Research in Germany] German Embassy, New Delhi. Retrieved 2010-28-08.]]
Germany has been the home of many famous inventors and [[engineer]]s, such as [[Johannes Gutenberg]], who is credited with the invention of [[movable type]] [[printing]] in Europe; [[Hans Geiger]], the creator of the [[Geiger counter]]; and [[Konrad Zuse]], who built the first fully automatic digital computer.Horst, Zuse. [http://www.epemag.com/zuse/ The Life and Work of Konrad Zuse]{{dead link|date=September 2010}} Everyday Practical Electronics (EPE) Online. Retrieved 2 January 2007. {{Wayback|url=http://www.epemag.com/zuse/|date =20070220074134|bot=DASHBot}}{{dead link|date=September 2010}} German inventors, engineers and industrialists such as [[Count]] [[Ferdinand von Zeppelin]], [[Otto Lilienthal]], [[Gottlieb Daimler]], [[Rudolf Diesel]], [[Hugo Junkers]] and [[Karl Benz]] helped shape modern automotive and air transportation technology.{{Cite encyclopedia|url=http://encarta.msn.com/encyclopedia_761576902_5/Automobile.html|title=Automobile|publisher=Microsoft Encarta Online Encyclopedia 2006|accessdate=2 January 2007|archiveurl=http://web.archive.org/web/20080420212258/http://encarta.msn.com/encyclopedia_761576902_5/Automobile.html|archivedate=20 April 2008}}[http://www.centennialofflight.gov/essay/Lighter_than_air/zeppelin/LTA8.htm The Zeppelin] U.S. Centennial of Flight Commission. Retrieved 2 January 2007.
[[Aerospace engineering|Aerospace engineer]] [[Wernher von Braun]] developed the [[V-2 rocket|first space rocket]] and later on was a prominent member of [[NASA]] and developed the [[Saturn V]] Moon rocket, which paved the way for the success of the US [[Apollo program]]. [[Heinrich Rudolf Hertz]]'s work in the domain of [[electromagnetic radiation]] was pivotal to the development of modern telecommunication.[http://www.itu.int/aboutitu/HistoricalFigures.html Historical figures in telecommunications.] International Telecommunication Union. 14 January 2004. Retrieved 2 January 2007.
With regard to triadic patents Germany is in third place after the USA and Japan. With more than 26,500 registrations for patents submitted to the European Patent Office, Germany is the leading European nation. [[Siemens]], [[Robert Bosch GmbH|Bosch]] and [[BASF]], with almost 5,000 registrations for patents between them in 2008, are among the Top 5 of more than 35,000 companies registering patents. Together with the USA and Japan, with regard to patents for nano, bio and new technologies Germany is one of the world’s most active nations. With around one third of triadic patents Germany leads the way worldwide in the field of vehicle emission reduction.[http://www.tatsachen-ueber-deutschland.de/en/education-and-research/main-content-07/industry-strongly-engaged-in-research.html Industry strongly engaged in research] Facts about Germany. Retrieved 2010-29-08.
===Science===
{{Main|Science and technology in Germany}}
[[File:Einstein 1921 by F Schmutzer.jpg|thumb|upright|[[Albert Einstein]]]]
Germany's achievements in sciences have been significant and [[research and development]] efforts form an integral part of the [[Economy of Germany|country's economy]]. Germany has been the home of some of the most prominent researchers in various scientific fields.[http://www.campus-germany.de/english/2.60.260.html Back to the Future: Germany – A Country of Research] German Academic Exchange Service (23 February 2005). Retrieved 8 December 2006.
The [[Nobel Prize]] has been awarded to [[Nobel laureates by country|103 German laureates]].[http://nobelprize.org/ Nobel Prize], Nobelprize.org. Retrieved 19 November 2009. For most of the 20th century, [[Evolution of Nobel Prizes by country|German laureates had more awards]] than those of any other nation, especially in the sciences (i.e., [[Nobel Prize in Physics|physics]], [[Nobel Prize in Chemistry|chemistry]], and [[Nobel Prize in Physiology or Medicine|physiology or medicine]]).[http://www.sciencenews.org/view/generic/id/63944/title/Swedish_academy_awards Swedish academy awards] ScienceNews web edition, 1 October 2010.National Science Nobel Prize shares 1901–2009 [http://www.idsia.ch/~juergen/sci.html by citizenship at the time of the award] and [http://www.idsia.ch/~juergen/scinat.html by country of birth]. From [[Jürgen Schmidhuber|J. Schmidhuber]] (2010), [http://www.idsia.ch/~juergen/nobelshare.html Evolution of National Nobel Prize Shares in the 20th Century] at [http://arxiv.org/abs/1009.2634 arXiv:1009.2634v1]
The work of [[Albert Einstein]] and [[Max Planck]] was crucial to the foundation of modern [[physics]], which [[Werner Heisenberg]] and [[Max Born]] developed further.Roberts, J. M. ''The New Penguin History of the World'', Penguin History, 2002. Pg. 1014. ISBN 0-14-100723-0. They were preceded by physicists such as [[Hermann von Helmholtz]], [[Joseph von Fraunhofer]], and [[Daniel Gabriel Fahrenheit]]. [[Wilhelm Conrad Röntgen]] discovered [[X-ray]]s, which are called ''Röntgenstrahlen'' (Röntgen-rays) in German and many other languages. This accomplishment made him the first winner of the [[Nobel Prize in Physics]] in 1901.[http://web.archive.org/web/20080402061044/http://www.history.com/encyclopedia.do?articleId=226611 The Alfred B. Nobel Prize Winners, 1901–2003] History Channel from ''The World Almanac and Book of Facts'' 2006. Retrieved 2 January 2007. Numerous significant [[mathematician]]s were born in Germany, including [[Carl Friedrich Gauss]], [[David Hilbert]], [[Bernhard Riemann]], [[Gottfried Leibniz]], [[Karl Weierstrass]] and [[Hermann Weyl]].
Important research institutions in Germany are the [[Max Planck Society]], the [[Helmholtz Association of German Research Centres|Helmholtz Association]] and the [[Fraunhofer Society]]. They are independently or externally connected to the university system and contribute to a considerable extent to the scientific output. The prestigious [[Gottfried Wilhelm Leibniz Prize]] is granted to ten scientists and academics every year. With a maximum of €2.5 million per award it is one of highest endowed research prizes in the world.[http://web.archive.org/web/20080621091621/http://www.dfg.de/en/research_funding/scientific_prizes/gw_leibniz_prize.html Gottfried Wilhelm Leibniz Prize], DFG. Retrieved 12 March 2007.
==Demographics==
[[File:Bevölkerungsdichte.gif|thumb|Population density in German states (left) and communities (right)]]
{{Main|Demographics of Germany|Germans|Social issues in Germany}}
With estimated 81.8 million inhabitants in January 2010, Germany is the most populous country in the European Union and ranks as the [[List of countries by population|15th largest country]] in the world in terms of population. Its [[List of sovereign states and dependent territories by population density|population density]] stands at 229.4 inhabitants per square kilometre.
The overall [[List of countries by life expectancy|life expectancy in Germany]] at birth is 79.9 years. The [[Total fertility rate|fertility rate]] of 1.4 children per mother, or 7.9 births per 1000 inhabitants in 2009, is one of the [[List of sovereign states and dependent territories by fertility rate|lowest in the world]].{{Cite web |url=http://www.destatis.de/jetspeed/portal/cms/Sites/destatis/Internet/DE/Presse/pm/2010/01/PD10__034__12641,templateId=renderPrint.psml|title=Durchschnittliche Kinderzahl 2008 in den neuen Ländern angestiegen|author=[[Federal Statistical Office of Germany|Destatis]] |language=German|accessdate=31 January 2010}} [[Death rates]] are continuously exceeding low-level [[birth rates]].{{Cite web | title = Demographic Transition Model | publisher=Barcelona Field Studies Centre | date = 27 September 2009 | url = http://geographyfieldwork.com/DemographicTransition.htm | accessdate = 23 May 2010}}
The [[Federal Statistical Office of Germany]] forecast that, as a result, the population will shrink to between 65 and 70 million by 2060 (depending on the level of net migration).The forecast of 65 million assumes a net migration of +100,000 per year; the forecast of 70 million assumes a net migration of +200,000 per year.
{{cite press release | publisher=[[Federal Statistical Office of Germany|Destatis]] | date = 2009-11-18| language = German|title=Pressemitteilung Nr.435 vom 18.11.2009:Im Jahr 2060 wird jeder Siebente 80 Jahre oder älter sein| trans_title = Press Release No. 435 of 18 November 2009: By 2006, every seventh person will be eighty or older| url=http://www.destatis.de/jetspeed/portal/cms/Sites/destatis/Internet/DE/Presse/pm/2009/11/PD09__435__12411,templateId=renderPrint.psml | accessdate = 2011-02-12| quote = Heute leben in Deutschland etwa 82 Millionen Menschen, 2060 werden es voraussichtlich nur noch 65 bis 70 Millionen sein. (Today there are 82 million people living in Germany; in 2006 it is forecast to be only 65 to 70 million.)}}
More details on the methodology, detailed tables broken down by age and sex, etc. are provided at
{{cite web| url = http://www.destatis.de/jetspeed/portal/cms/Sites/destatis/Internet/DE/Presse/abisz/Bevoelkerungsvorausberechnung,templateId=renderPrint.psml| title = Bevölkerungsentwicklung in Deutschland bis 2060| trans_title = Population development in Germany till 2060| date = 2009-11-18| publisher = [[Statistisches Bundesamt]] | language = German| accessdate = 2011-02-15}}
[[File:Pyramide Allemagne.PNG|thumb|left|Germany's [[population pyramid]] in 2005]]
Over 99% of those of age 15 and above are estimated to be able to read and write. A growing number of inhabitants are functionally illiterate. Illiteracy rates of youngsters vary by ethnic group and parents socio-economic class.
[[Germans]] make up 91% of the population of Germany. As of 2004, about seven million foreign citizens were registered in Germany, and 19% of the country's residents were of foreign or partially foreign descent (including persons descending or partially descending from [[ethnic German]] repatriates), 96% of whom lived in Western Germany or Berlin. It is estimated by the [[Federal Statistical Office of Germany]] that nearly 30% of Germans aged 5 years and younger have at least one parent who was born abroad.
===Migration===
[[File:Population of German territories 1800 - 2000.JPG|thumb|Population of German territories 1800–2000 and [[immigrant]] population from 1975–2000]]
The [[United Nations Population Fund]] lists Germany as host to the [[List of countries by immigrant population|third-highest number of international migrants worldwide]], about 5% or 10 million of all 191 million migrants.[http://www.unfpa.org/publications/detail.cfm?ID=294&filterListType= State of World Population 2006] United Nations Population Fund. 2006. Retrieved 1 January 2007. As a consequence of [[Immigration to Germany|restrictions]] to Germany's formerly rather unrestricted laws on [[right of asylum|asylum]] and immigration, the number of immigrants seeking asylum or claiming German ethnicity (mostly from the former Soviet Union) has been declining steadily since 2000."[http://www.focus-migration.de/Germany.1509.0.html?&L=1 Focus-Migration: Germany]".
In 2009, 20% of the population had immigrant roots, the highest since records were kept since 1945.{{Cite news | title=20% of Germans have immigrant roots | newspaper=Burlington Free Press | location=Burlington, Vermont, USA | pages= 4A | date=15 July 2010 }} {{As of|2008}}, the largest national group of people with a migrant background was from Turkey (2.5 million), followed by Italy (776,000) and Poland (687,000).{{Cite web|url=http://www.destatis.de/jetspeed/portal/cms/Sites/destatis/Internet/DE/Content/Statistiken/Bevoelkerung/MigrationIntegration/Migrationshintergrund/Tabellen/Content100/MigrationshintergrundStaatsangehoerigkeit,templateId=renderPrint.psml|title=Bevölkerung nach Migrationshintergrund|accessdate=26 March 2010|publisher=German Federal Statistical Office|language=German}}
Large numbers of people with full or significant German ancestry are found in the
United States (50 million),{{Cite web |url=http://factfinder.census.gov/servlet/IPTable?_bm=y&-reg=ACS_2005_EST_G00_S0201:535;ACS_2005_EST_G00_S0201PR:535;ACS_2005_EST_G00_S0201T:535;ACS_2005_EST_G00_S0201TPR:535&-qr_name=ACS_2005_EST_G00_S0201&-qr_name=ACS_2005_EST_G00_S0201PR&-qr_name=ACS_2005_EST_G00_S0201T&-qr_name=ACS_2005_EST_G00_S0201TPR&-ds_name=ACS_2005_EST_G00_&-TABLE_NAMEX=&-ci_type=A&-redoLog=false&-charIterations=047&-geo_id=01000US&-format=&-_lang=en|coauthors=United States Census Bureau|title=U.S. Census Bureau, 2005 American Community Survey|accessdate=15 April 2007}} The 1990 census gives 57.9 million, or 23.3% of the U.S. population. [[Brazil]] (5 million){{pt icon}} [http://www.passeiweb.com/na_ponta_lingua/sala_de_aula/geografia/geografia_do_brasil/demografia_imigracoes/brasil_imigracoes_alemanha A Imigração Alemã no Brasil] Deutsche Welle. Retrieved 19 November 2009. and Canada (3 million).[http://www12.statcan.ca/english/census01/products/highlight/ETO/Table1.cfm?Lang=E&T=501&GV=1&GID=0 2001 Canadian Census] gives 2,742,765 total respondents stating their ''ethnic origin'' as partly German, with 705,600 stating "single-ancestry". About 3 million "Aussiedler"—ethnic Germans, mainly from Eastern Europe and the former Soviet Union—have resettled in Germany since 1987.[http://www.migrationinformation.org/Feature/display.cfm?ID=201 "Fewer Ethnic Germans Immigrating to Ancestral Homeland"]. ''Migration Information Source.'' February 2004
===Urbanisation===
{{See also|Metropolitan Regions in Germany|List of cities in Germany|List of cities in Germany with more than 100,000 inhabitants}}
Germany has a number of large cities; the most populous are: [[Berlin]], [[Hamburg]], [[Munich]], [[Cologne]], [[Frankfurt]], and [[Stuttgart]]. The largest [[conurbation]] is the [[Rhine-Ruhr]] region (12 million), including [[Düsseldorf]] (the capital of [[North Rhine-Westphalia]]), [[Cologne]], [[Essen]], [[Dortmund]], [[Duisburg]], and [[Bochum]].
{| width="97%" align="center" style="border: 1px solid darkgray; background: #f9f9f9"
!colspan=9 style="padding:0.3em 0; font-size:120%;" | Largest cities of Germany
|-
| rowspan=11 align=center valign=top | [[File:Cityscapeberlin2006.JPG|border|160px|Berlin]]
[[Berlin]]
[[File:Hamburg-090612-0164-DSC 8261-Hafen-von-Jugendherberge.jpg|border|160px|Hamburg]]
[[Hamburg]]
[[File:München Panorama.JPG|border|160px|Munich]]
[[Munich|Munich (München)]]
! style="text-align:center; background:#f5f5f5;"| Rank
! style="text-align:left; background:#f5f5f5;"| City
! style="text-align:right; background:#f5f5f5;"| Population
| rowspan=11 width="50" |
! style="text-align:center; background:#f5f5f5;"| Rank
! style="text-align:left; background:#f5f5f5;"| City
! style="text-align:right; background:#f5f5f5;"| Population
| rowspan=11 align=center valign=top | [[File:Köln Panorama.jpg|border|160px|Cologne]]
[[Cologne|Cologne (Köln)]]
[[File:Hauptwache Frankfurt am Main.jpg|border|160px|Frankfurt am Main]]
[[Frankfurt am Main|Frankfurt]]
[[File:Stg-schlossplatz-kunstmuseum.jpg|border|160px|Stuttgart]]
[[Stuttgart]]
|-
| align=center style="background:#f0f0f0;" | 1 || align=left | [[Berlin]] || align=right | 3,442,675
| align=center style="background:#f0f0f0;" | 11 || align=left | [[Hannover]] || align=right | 520,966
|-
| align=center style="background:#f0f0f0;" | 2 || align=left | [[Hamburg]] || align=right | 1,774,224
| align=center style="background:#f0f0f0;" | 12 || align=left | [[Leipzig]] || align=right | 518,862
|-
| align=center style="background:#f0f0f0;" | 3 || align=left | [[Munich]] || align=right | 1,330,440
| align=center style="background:#f0f0f0;" | 13 || align=left | [[Dresden]] || align=right | 517,052
|-
| align=center style="background:#f0f0f0;" | 4 || align=left | [[Cologne]] || align=right | 998,105
| align=center style="background:#f0f0f0;" | 14 || align=left | [[Nuremberg]] || align=right | 503,673
|-
| align=center style="background:#f0f0f0;" | 5 || align=left | [[Frankfurt am Main|Frankfurt]] || align=right | 671,927
| align=center style="background:#f0f0f0;" | 15 || align=left | [[Duisburg]] || align=right | 491,931
|-
| align=center style="background:#f0f0f0;" | 6 || align=left | [[Stuttgart]] || align=right | 601,646
| align=center style="background:#f0f0f0;" | 16 || align=left | [[Bochum]] || align=right | 376,319
|-
| align=center style="background:#f0f0f0;" | 7 || align=left | [[Düsseldorf]] || align=right | 586,217
| align=center style="background:#f0f0f0;" | 17 || align=left | [[Wuppertal]] || align=right | 351,050
|-
| align=center style="background:#f0f0f0;" | 8 || align=left | [[Dortmund]] || align=right | 581,308
| align=center style="background:#f0f0f0;" | 18 || align=left | [[Bielefeld]] || align=right | 323,084
|-
| align=center style="background:#f0f0f0;" | 9 || align=left | [[Essen]] || align=right | 576,259
| align=center style="background:#f0f0f0;" | 19 || align=left | [[Bonn]] || align=right | 319,841
|-
| align=center style="background:#f0f0f0;" | 10 || align=left | [[Bremen]] || align=right | 547,685
| align=center style="background:#f0f0f0;" | 20 || align=left | [[Mannheim]] || align=right | 311,969
|-
| colspan=9 align=center | Source: {{cite web |title=Mitglieder des Deutschen Städtetages |publisher=Deutschen Städtetag |date=31 December 2009 |accessdate=18 February 2011 |url=http://www.staedtetag.de/10/staedte/nach_einwohner/index.html |language=German}}
|}
===Languages===
{{Main|Languages of Germany}}
[[File:Knowledge German EU map.png|thumb|upright|Knowledge of German in European countries]]
German is the official and predominant spoken language in Germany. It is one of 23 official languages in the European Union, and one of the three [[working language]]s of the [[European Commission]], along with English and French. Recognised native minority languages in Germany are Danish, [[Sorbian languages|Sorbian]], [[Romany language|Romany]], and [[Frisian languages|Frisian]]. They are officially protected by the [[ECRML]]. The most used immigrant languages are [[Turkish language|Turkish]], [[Kurdish language|Kurdish]], Polish, the [[Balkan languages]], and Russian.
Standard German is a [[West Germanic languages|West Germanic language]] and is closely related to and classified alongside English, [[Dutch language|Dutch]], and the [[Frisian languages]]. To a lesser extent, it is also related to the [[East Germanic languages|East]] (extinct) and [[North Germanic languages]]. Most German vocabulary is derived from the Germanic branch of the Indo-European language family.{{Cite web|title=Many tongues, one family. Languages in the European Union|publisher=[[Europa (web portal)]] |author=[[European Commission]] |year=2004|url=http://ec.europa.eu/publications/booklets/move/45/en.pdf|format=PDF |accessdate=3 February 2007}} Significant minorities of words are derived from [[Latin]] and Greek, with a smaller amount from French and most recently English (known as [[Denglisch]]). German is written using the Latin alphabet. In addition to the 26 standard letters, German has three vowels with [[Umlaut (diacritic)|Umlauts]], namely ä, ö, and ü, as well as the [[Eszett]] or ''scharfes S'' (''sharp s'') which is written "ß".
{{multiple image
| align = left
| width2 = 170
| image2 = De-Faust - Der Tragödie erster Teil-Prolog und Auf dem Theater-wikisource.ogg
| caption2 = Spoken German in [[Goethe's Faust]]
| alt2 = Goethe´s German
| width1 = 100
| image1 = Faust spricht mit dem Erdgeist, Margret Hofheinz-Döring, Öl, 1969 (WV-Nr.4385).jpg
| caption1 =
| alt1 = Faust
}}
German [[dialect]]s are distinguished from [[variety (linguistics)|varieties]] of [[standard German]]. German dialects are traditional local varieties and are traced back to the different German tribes. Many of them are not easily understandable to a speaker of standard German, since they often differ in [[lexicon]], [[phonology]], and [[syntax]].
Around the world, German has approximately 100 million [[First language|native speakers]] and also about 80 million non-native speakers.{{Cite book|authorlink=National Geographic|title=National Geographic Collegiate Atlas of the World|publisher=R.R Donnelley & Sons Company|year=2006|month=April|location=Willard, Ohio|pages=257–270|isbn=Regular:0-7922-3662-9, 978-0-7922-3662-7. Deluxe:0-7922-7976-X, 978-0-7922-7976-1}} German is the main language of about 90 million people (18%) in the EU. 67% of German citizens claim to be able to communicate in at least one foreign language, 27% in at least two languages other than their own.{{Cite web|title=Special Eurobarometer 243: Europeans and their Languages (Survey)|publisher=[[Europa (web portal)]] |author=[[European Commission]]|year=2006|url=http://ec.europa.eu/public_opinion/archives/ebs/ebs_243_en.pdf| format=PDF |accessdate=3 February 2007}}
{{Cite web|title=Special Eurobarometer 243: Europeans and their Languages (Executive Summary)|publisher=[[Europa (web portal)]] |author=[[European Commission]]|year=2006|url=http://ec.europa.eu/public_opinion/archives/ebs/ebs_243_sum_en.pdf|format=PDF|doi=|accessdate=3 February 2007}}
===Religion===
{{Main|Religion in Germany}}
[[File:Colognecathedralatnight.JPG|thumb|upright|The [[Cologne Cathedral]] at the [[Rhine]] river is a [[UNESCO World Heritage Site]].]]
According to organisational reporting based on projections in 2008 about 34.1% Germans have [[irreligion|no registered religious denomination]]. According to a poll by Der Spiegel magazine, 45% believe in God, and just a quarter in Jesus Christ.{{Cite web|url=http://www.adherents.com/adhloc/Wh_112.html |title=By Location |publisher=Adherents.com |date= |accessdate=26 September 2010}}
Christianity is the largest religion in Germany, with around 49.4 million adherents (62.8%) in 2008{{de icon}} {{Cite web|url=http://www.ekd.de/statistik/mitglieder.html |title=EKD-Statistik: Christen in Deutschland 2007 |publisher=Evangelische Kirche in Deutschland |date= |accessdate=13 February 2010}} of which 24.5 million are [[Protestantism|Protestants]] (29.9%) belonging to the EKD and 24.9 million are Catholics (30.0%) in 2008, the remainder belong to small denominations (each (considerably ) less than 0.5% of the German population).[http://fowid.de/fileadmin/datenarchiv/Religionszugehoerigkeit_Bevoelkerung__1950-2008.pdf Konfessionen in Deutschland](German), fowid. Retrieved 2010, 9 September-09. The second largest religion is [[Islam in Germany|Islam]] with an estimated 3.8 to 4.3 million adherents (4.6 to 5.2%) followed by [[Buddhism]] and Judaism, both with around 200,000 adherents (0.3%). [[Hinduism]] has some 90,000 adherents (0.1%) and [[Sikhism]] 75,000 (0.1%). All other religious communities in Germany have fewer than 50,000 (<0.1%) adherents.
{{bar box
|title=Religion in Germany (2008)
|titlebar=#ddd
|float=left
|bars=
{{bar percent|[[Irreligion|No Religion]]|Yellow|34.1}}
{{bar percent|[[Roman Catholicism]]|Orange|30.0}}
{{bar percent|[[Protestantism]]|MediumAquamarine|29.9}}
{{bar percent|[[Islam]]|#f99|4.0}}
{{bar percent|[[Eastern Orthodox Church|Orthodox Christianity]]|DarkGray|1.6}}
{{bar percent|[[Judaism]]|Blue|0.2}}
{{bar percent|[[Buddhism]]|Black|0.2}}
}}
Protestantism is concentrated in the north and east and [[Roman Catholicism in Germany|Roman Catholicism]] is concentrated in the south and west. Pope [[Benedict XVI]], was born in [[Bavaria]]. [[irreligion|Non-religious]] people, including atheists and agnostics might make as many as 55%, and are especially numerous in the former [[German Democratic Republic|East Germany]] and major metropolitan areas.{{de icon}} [http://www.remid.de/remid_info_zahlen.htm Religionen in Deutschland: Mitgliederzahlen] Religionswissenschaftlicher Medien- und Informationsdienst; 31 October 2009. Retrieved 19 November 2009.
Of the roughly 4 million Muslims, most are [[Sunni Islam|Sunnis]] and [[Alevites]] from [[Turkey]], but there are a small number of [[Shia Islam|Shi'ites]] and other denominations.{{Cite book | title = Muslimisches Leben in Deutschland |trans_title= Muslim Life in Germany | url = http://www.bmi.bund.de/cae/servlet/contentblob/566008/publicationFile/31710/vollversion_studie_muslim_leben_deutschland_.pdf;jsessionid=6B8CD26E2AC179111AF4F75650B84B1A | format = PDF | accessdate = 9 September 2010 | year = 2009 | month = June | publisher=Federal Office for Migration and Refugees (German: Bundesamt für Migration und Flüchtlinge), an agency of the [[Federal Ministry of the Interior (Germany)]] | location = [[Nuremberg]] | language = German | isbn =978-3-9812115-1-1| page = 80 | chapter = Chapter 2: Wie viele Muslime leben in Deutschland?|trans_chapter=How many Muslims live in Germany?| quote =Demnach leben in Deutschland zwischen 3,8 und4,3 Millionen Muslime [. . .] beträgt der Anteil der Muslime an der Gesamtbevölkerungzwischen 4,6 und 5,2 Prozent. Rund 45 Prozent der in Deutschland lebenden Muslime sind deutsche Staatsangehörige,rund 55 Prozent haben eine ausländische Staatsangehörigkeit. }}{{Cite book | title = Muslimisches Leben in Deutschland |trans_title= Muslim Life in Germany | url = http://www.bmi.bund.de/cae/servlet/contentblob/566008/publicationFile/31710/vollversion_studie_muslim_leben_deutschland_.pdf;jsessionid=6B8CD26E2AC179111AF4F75650B84B1A | format = PDF | accessdate = 9 September 2010 | year = 2009 | month = June | publisher=Federal Office for Migration and Refugees (German: Bundesamt für Migration und Flüchtlinge), an agency of the [[Federal Ministry of the Interior (Germany)]] | location = [[Nuremberg]] | language = German | isbn =978-3-9812115-1-1| page= 97 | chapter = Chapter 2: Wie viele Muslime leben in Deutschland?|trans_chapter=How many Muslims live in Germany?| quote =Der Anteil der Sunniten unter den in den Haushalten lebenden Muslimen beträgt 74 Prozent }}
* 74.1% [[Sunni Islam|Sunni]]
* 12.7% [[Alevi]]te
* 7.1% [[Shia Islam|Schiite]]
* 1.7% [[Ahmadiyya|Ahmadi]]
* 0.3% [[Ibadi]]
* 0.1% [[Sufism|Sufi]]/mystic
* 4.0% other
1.6% of the country's overall population declare themselves [[Orthodox Christians]], [[Serbs]] and [[Greeks]] being the most numerous. Germany has Europe's third-largest [[History of the Jews in Germany|Jewish population]] (after France and the United Kingdom).Blake, Mariah. [http://www.csmonitor.com/2006/1110/p25s02-woeu.html In Nazi cradle, Germany marks Jewish renaissance] [[Christian Science Monitor]]. 10 November 2006. Retrieved 30 November 2006. In 2004, twice as many Jews from former [[Soviet Union|Soviet]] republics settled in Germany as in [[Israel]], bringing the total Jewish population to more than 200,000, compared to 30,000 prior to [[German reunification]]. Large cities with significant Jewish populations include Berlin, [[Frankfurt]] and [[Munich]].[http://www.eurojewcong.org/ejc/news.php?id_article=81 The Jewish Community of Germany] European Jewish Congress. Retrieved 30 November 2006. Around 250,000 active [[Buddhism|Buddhists]] live in Germany; 50% of them are Asian immigrants.{{de icon}} [[Die Zeit]] 12/07, page 13
According to the [[Eurobarometer]] poll 2005, 47% of German citizens agreed with the statement "I believe there is a God", whereas 25% agreed with "I believe there is some sort of spirit or life force" and 25% said "I do not believe there is any sort of spirit, god, or life force".{{Cite web|url=http://ec.europa.eu/public_opinion/archives/ebs/ebs_225_report_en.pdf|title=Eurobarometer on Social Values, Science and technology 2005 (page 11)|accessdate=5 May 2007|format=PDF}}
===Education===
{{Main|Education in Germany}}
[[File:Kindergartenfrankfurt.jpg|thumb|Kindergarten in [[Hessen]]]]
Responsibility for educational oversight in Germany lies primarily with the [[States of Germany|federated states]] individually, whilst the federal government only has a minor role. Optional [[kindergarten]] education is provided for all children between three and six years old, after which school attendance is [[compulsory education|compulsory]] for at least nine years. Primary education usually lasts for four years and public schools are not stratified at this stage.{{PDFlink|[http://lcweb2.loc.gov/frd/cs/profiles/Germany.pdf Country profile: Germany]|177 KB}} U.S. Library of Congress. December 2005. Retrieved 4 December 2006. In contrast, secondary education includes three traditional types of schools based on a pupil's ability as determined by teacher recommendations: the ''[[Gymnasium (school)|Gymnasium]]'' enrolls the most gifted children and prepares students for university studies, and attendance lasts eight or nine years depending on the state; the ''[[Realschule]]'' has a broader range of emphasis for intermediate students and lasts six years; the ''[[Hauptschule]]'' prepares pupils for vocational education.
[[File:Heidelberg Universitätsbibliothek 2003.jpg|thumb|left|The [[University of Heidelberg]] was established in 1386.]]
Since the 1960s, a reform movement attempted to unify secondary education in a ''Gesamtschule'' ([[comprehensive school]]). However, instead of overcoming the stratification, Gesamtschule just became a fourth type of secondary school. Since about 2000, several West German Länder simplified their school system to two or three tiers. Motives were: The example of Eastern Germany where in the 1990s, following reunification, a two-tier school system was established; mediocre scores in the [[Programme for International Student Assessment]] (PISA), first published in 2001, prompted a nation-wide debate about the school system, and in particular about the social selectivity of early stratification;In PISA 2000, Germany was the country with the highest correlation between test score and social background index. In the following test periods, this result could not be replicated. having low standards of achievement, inner-city Hauptschulen were increasingly considered dysfunctional by some people; outside the main metropolitan areas, the population is shrinking, so that according to some people it becomes increasingly impractical to maintain a three- or four-tier school system.
A special system of apprenticeship called ''Duale Ausbildung'' ("dual education") allows pupils in [[vocational training]] to learn in a company as well as in a state-run vocational school.
To enter a [[List of universities in Germany|university in Germany]], high school students are generally required to take the ''[[Abitur]]'' examination, which is similar to [[Advanced Level (UK)|A-levels in the UK]] and typically done at the age of 18 or 19. However, students possessing a diploma from a [[vocational school]] may also apply for [[matriculation]] in certain subjects. Germany's universities are recognised internationally, indicating the high education standards in the country. In the [[Academic Ranking of World Universities|ARWU]] ranking for 2008, six of the top 100 universities in the world are in Germany, and 18 in the top 200.{{Cite web|title=Top 100 World Universities|publisher=[[Academic Ranking of World Universities|ARWU]]|url=http://www.arwu.org/rank2008/ARWU2008_A(EN).htm|archiveurl=http://web.archive.org/web/20080822124509/http://www.arwu.org/rank2008/ARWU2008_A(EN).htm|archivedate=22 August 2008|accessdate=14 March 2009}} Nearly all German universities are public (i.e. non-private) institutions, charging tuition fees ranging from €50–500 per semester for each student.[http://www.daad.de/deutschland/studium/studienplanung/00493.en.html#headline_0_1 Tuition Fees in Germany] German Academic Exchange Service. Retrieved 30 November 2006.
===Health===
{{Main|Health in Germany}}
[[File:UniKlinik (Freiburg) 4.jpg|thumb|Department of Surgery in the [[University Medical Center Freiburg]]]]
Germany has the world's oldest [[universal health care]] system, with origins dating back to [[Otto von Bismarck]]'s [[Otto von Bismarck#Bismarck's social legislation|Social legislation]] in 1883. Currently the population is covered by a basic health insurance plan provided by statute, which provides a standard level of coverage. The remainder opt for private health insurance, which frequently offers additional benefits. According to the [[World Health Organization]], Germany's [[health care system]] was 77% government-funded and 23% privately funded as of 2004.[http://www.who.int/whosis/database/core/core_select.cfm World Health Organization Statistical Information System: Core Health Indicators]. Retrieved 13 November 2010. In 2005, Germany spent 10.7% of GDP on health care, or US$3,628 per capita.
Germany ranked 20th in the world in [[life expectancy]] with [[List of countries by life expectancy|76,5 years for men and 82,1 years for woman]]. It had a very low [[infant mortality rate]] (4.3 per 1,000 [[Live birth (human)|live births]]), and it was tied for eighth place in the number of practicing physicians, at per 1,000 people (3.3).
In 2002 the top diagnosis for male patients released from the hospital was [[heart disease]], followed by [[Alcoholism|alcohol-related disorders]] and [[hernias]]. For women, the top diagnoses related to [[pregnancies]], breast cancer, and heart weakness. At the end of 2004, some 44,000 Germans, or less than 0.1 percent of the population, were infected with [[HIV/AIDS]]. According to a 2003 survey, 37 percent of adult males and 28 percent of adult females in Germany are smokers. A 2007 study shows Germany has the highest number of overweight people in Europe.{{Cite news|title=Topping the EU Fat Stats, Germany Plans Anti-Obesity Drive|url=http://www.dw-world.de/dw/article/0,,2449356,00.html|publisher=Deutsche Welle|date=20 April 2007|accessdate=25 June 2010}}{{Cite news|title=Germany launches obesity campaign|url=http://news.bbc.co.uk/2/hi/health/6639227.stm|publisher=BBC|date=9 May 2007|accessdate =25 June 2010}}
==Culture==
{{Main|Culture of Germany}}
[[File:Beethoven.jpg|thumb|left|upright|[[Ludwig van Beethoven]] (1770–1827), composer]]
German culture began long before the rise of Germany as a [[nation-state]] and spanned the entire German-speaking world. From its roots, culture in Germany has been shaped by major intellectual and popular currents in Europe, both religious and [[secularism|secular]]. As a result, German artistic achievements have to be seen in a larger framework of European [[high culture]], Western [[popculture]], and globally connected [[subculture]].{{Cite encyclopedia|url=http://encarta.msn.com/encyclopedia_761576917_4/Germany.html|title=Federal Republic of Germany: Culture|publisher=Encarta Online Encyclopedia 2006|accessdate=5 November 2010|archiveurl=http://web.archive.org/web/20080328162029/http://encarta.msn.com/encyclopedia_761576917_4/Germany.html|archivedate=28 March 2008}}[http://www.scribd.com/doc/30887674/Modern-German-Culture The Cambridge Companion to Modern German Culture]. Retrieved 9 November 2010. Historically Germany has been called ''Das Land der Dichter und Denker'' (the land of poets and thinkers).Wasser, Jeremy. [http://www.spiegel.de/international/0,1518,410135,00.html Spätzle Westerns] Spiegel Online International. 6 April 2006. Retrieved 6 December 2006.
In Germany, the federated states are in charge of the cultural institutions. There are 240 subsidised theatres, hundreds of symphonic orchestras, thousands of museums and over 25,000 libraries spread over the 16 states. These cultural opportunities are enjoyed by many millions: there are over 91 million German museum visits every year; annually, 20 million go to theatres and operas; while 3.6 million listen to the great symphonic orchestras.[http://www.study-in-germany.de/english/4.303.659.html Unbelievable Multitude] www.study-in-germany.de Retrieved 17 October 2008.
[[File:Berlin night.jpg|thumb|Berlin is a centre of arts, nightlife, music, architecture and festivals]]
The UNESCO inscribed 33 properties in [[Table of World Heritage Sites by country|Germany on the World Heritage List]].[http://www.worldheritagesite.org/countries/germany.html World Heritage Sites in Germany] www.worldheritagesite.org Retrieved 3 October 2010.
Numerous German painters have enjoyed international prestige through their work in diverse artistic styles. [[Hans Holbein the Younger]], [[Matthias Grünewald]], and [[Albrecht Dürer]] were important artists of the [[Renaissance]], [[Caspar David Friedrich]] of [[Romanticism]], and [[Max Ernst]] of [[Surrealism]]. Architectural contributions from Germany include the [[Carolingian architecture|Carolingian]] and [[Ottonian architecture|Ottonian styles]], which were important precursors of [[Romanesque architecture|Romanesque]]. The region later became the site of significant works in styles such as [[Gothic architecture|Gothic]], [[Renaissance architecture|Renaissance]] and [[Baroque architecture|Baroque]]. Germany was particularly important in the early [[modern architecture|modern movement]], especially through the [[Bauhaus]] movement founded by [[Walter Gropius]]. [[Ludwig Mies van der Rohe]], also from Germany, became one of the world's most renowned architects in the second half of the 20th century. The glass façade skyscraper was his idea.2006 A Dictionary of Architecture and Landscape Architecture (Paperback) , Second (in English) , Oxford University Press, 880. ISBN 0-19-860678-8
===Literature===
{{Main|German literature}}
German literature can be traced back to the Middle Ages and the works of writers such as [[Walther von der Vogelweide]] and [[Wolfram von Eschenbach]]. Various German authors and poets have won great renown, including [[Johann Wolfgang von Goethe]] and [[Friedrich Schiller]]. The collections of folk tales published by the [[Brothers Grimm]] popularised [[German folklore]] on an international level.
Influential authors of the 20th century include [[Thomas Mann]], [[Bertolt Brecht]], [[Hermann Hesse]], [[Heinrich Böll]], and [[Günter Grass]].{{Cite web|url=http://nobelprize.org/nobel_prizes/literature/articles/espmark/index.html|title=The Nobel Prize in Literature|publisher=Nobelprize.org|date=3 December 1999|author=Kjell Espmark|accessdate=14 August 2006}}
{| class="wikitable" style="text-align:right; margin-right:50px;"
|-
! align=center |[[Johann Wolfgang von Goethe|Johann Wolfgang v. Goethe]]
(1749–1832)
! align=center |[[Friedrich Schiller]]
(1759–1805)
! align=center |[[Brothers Grimm]]
(1785–1863)
! align=center |[[Thomas Mann]]
(1875–1955)
! align=center |[[Hermann Hesse]]
(1877–1962)
|-
| align=left | [[File:Johann Heinrich Wilhelm Tischbein 007.jpg|190px]] || align=left | [[File:Gerhard von Kügelgen 001.jpg|129px]]|| align=left | [[File:Grimm1.jpg|125px]]|| align=left | [[File:Thomas Mann 1929.jpg|106px]] || align=left | [[File:Hermann Hesse 1927 Photo Gret Widmann.jpg|110px]]
|}
===Philosophy===
{{Main|German philosophy}}
[[File:Immanuel Kant (painted portrait).jpg|thumb|upright|[[Immanuel Kant]] (1724–1804)]]
Germany's influence on philosophy is historically significant and many notable German philosophers have helped shape [[Western philosophy]] since the Middle Ages. The rise of the modern natural sciences and the related decline of religion raised a series of questions, which recur throughout German philosophy, concerning the relationships between knowledge and faith, reason and emotion, and scientific, ethical, and artistic ways of seeing the world.
[[Gottfried Leibniz]]'s contributions to [[rationalism]]; the establishment of classical [[German idealism]] by [[Immanuel Kant]], [[Johann Gottlieb Fichte]], [[Georg Wilhelm Friedrich Hegel]] and [[Friedrich Wilhelm Joseph Schelling]]; [[Arthur Schopenhauer]]'s composition of metaphysical pessimism; [[Karl Marx]]'s and [[Friedrich Engels]]' formulation of [[communist theory]]; [[Friedrich Nietzsche]]'s development of [[perspectivism]]; [[Gottlob Frege]]'s contributions to the dawn of [[analytic philosophy]]; [[Martin Heidegger]]'s works on Being; and the development of the [[Frankfurt school]] by [[Max Horkheimer]], [[Theodor Adorno]], [[Herbert Marcuse]] and [[Jürgen Habermas]] have been particularly influential.
In the 21st century Germany has been an important country for the development of contemporary analytic philosophy in continental Europe, along with France, Austria, Switzerland and the Scandinavian countries.Searle, John. (1987). ''The Blackwell Companion to Philosophy'', "Introduction". Wiley-Blackwell.
===Cinema===
{{Main|Cinema of Germany}}
[[File:Derblaueengel.jpg|thumb|upright|[[Marlene Dietrich]] in ''[[The Blue Angel]]'' in 1930, Germany's first major film with sound]]
German cinema dates back to the very early years of the medium with the work of [[Max Skladanowsky]]. It was particularly influential during the years of the Weimar Republic with [[German expressionism|German expressionists]] such as [[Robert Wiene]] and [[Friedrich Wilhelm Murnau]]. Austrian-based director [[Fritz Lang]], who became a German citizen in 1926 and whose career flourished in the pre-war German film industry, is said to have been a major influence on Hollywood cinema. His silent movie [[Metropolis (film)|Metropolis]] (1927) is referred to as the birth of modern Science Fiction movies.
In 1930 Austrian-American [[Josef von Sternberg]] directed ''[[The Blue Angel]]'', which was the first major German [[sound film]] and it brought world fame to actress [[Marlene Dietrich]].{{Cite book |last= Bordwell |first= David |authorlink= David Bordwell |coauthors= Kristin Thompson|title= Film History: An Introduction|origdate= |origyear= 1994 |edition= 2nd |year= 2003 |publisher=McGraw-Hill |location= New York City |isbn= 978-0-07-115141-2 |page= 204 |chapter= The Introduction of Sound}} Impressionist documentary [[Berlin: Symphony of a Great City]], directed by [[Walter Ruttmann]], is a prominent example of the city symphony genre. The Nazi era produced mostly propaganda films although the work of [[Leni Riefenstahl]] still introduced new aesthetics to film.[http://www.filmbug.com/db/343340 Leni Riefenstahl], Filmbug Movie Stars. Retrieved 13 April 2007.
[[File:Berlinale2007.jpg|thumb|left|The Berlinale Palast during the [[Berlin Film Festival]] in February 2007]]
During the 1970s and 80s, [[New German Cinema]] directors such as [[Volker Schlöndorff]], [[Werner Herzog]], [[Wim Wenders]], and [[Rainer Werner Fassbinder]] put West German cinema back on the international stage with their often provocative films.[http://www.fassbinderfoundation.de/node.php/en/home Rainer Werner Fassbinder], Fassbinder Foundation. Retrieved 13 April 2007.
More recently, films such as ''[[Good Bye Lenin!]]'' (2003), ''[[Head-On|Gegen die Wand (Head-on)]]'' (2004), ''[[Der Untergang|Der Untergang (Downfall)]]'' (2004), and ''[[Der Baader Meinhof Komplex]]'' (2008) have enjoyed international success.
The [[Academy Award for Best Foreign Language Film]] went to the German production ''[[The Tin Drum (film)|Die Blechtrommel (The Tin Drum)]]'' in 1979, to ''[[Nowhere in Africa]]'' in 2002, and to ''[[The Lives of Others|Das Leben der Anderen (The Lives of Others)]]'' in 2007.[http://www.imdb.com/title/tt0405094/awards Awards:Das Leben der Anderen], IMDb. Retrieved 13 April 2007. Among the most famous German actors are Marlene Dietrich, [[Klaus Kinski]], [[Hanna Schygulla]], [[Armin Mueller-Stahl]], [[Jürgen Prochnow]], and [[Thomas Kretschmann]].
The [[Berlin Film Festival]], held annually since 1951, is one of the world's foremost film festivals. An international jury places emphasis on representing films from all over the world and awards the winners with the Golden and Silver Bears.''[http://www.fiapf.org/pdf/2006accreditedFestivalsDirectory.pdf 2006 FIAPF accredited Festivals Directory], International Federation of Film Producers Associations, retrieved on 11 December 2006.'' The annual [[European Film Award]]s ceremony is held every second year in the city of Berlin, where the [[European Film Academy]] (EFA) is located. The [[Babelsberg Studios]] in Potsdam are the oldest large-scale film studios in the world and a centre for international film production.
===Music===
{{Main|Music of Germany}}
Germany claims some of the world's most renowned classical music composers, including [[Ludwig van Beethoven]], [[Johann Sebastian Bach]], [[Johannes Brahms]], and [[Richard Wagner]]. As of 2006, Germany is the fifth largest music market in the world and has influenced popular music through artists such as [[Kraftwerk]], [[Boney M.]], [[Nico]], [[Nina Hagen]], [[Scorpions (band)|Scorpions]], [[Toten Hosen]], [[Tokio Hotel]], [[Rammstein]] and [[Paul van Dyk]].[http://web.archive.org/web/20070205193206/http://news.warez.com/p2pnet/music-market-worth-us32-billion.html Music market worth US$ 32 billion] P2pnet.net 7 April 2004. Retrieved 7 December 2006.
{| class="wikitable" style="text-align:right; margin-right:50px;"
|-
! align=center |[[J.S.Bach]]
[[Toccata and Fugue in D minor, BWV 565|Toccata und Fuge]]
! align=center |[[Ludwig van Beethoven|L.v. Beethoven]]
[[Symphony No. 5 (Beethoven)|Symphonie 5 c-moll]]
! align=center |[[Richard Wagner|R. Wagner]]
[[Die Walküre]]
! align=center |[[Kraftwerk]]
[[Computer World|Computerwelt Pt. 2]]
! align=center |[[Modern Talking]]
[[Brother Louie (Modern Talking song)|Brother Louie]]
! align=center |[[Rammstein]]
[[Ohne dich]]
|-
| style="text-align:left; background:#dcdcdc;"| [[File:Toccata et Fugue BWV565.ogg|110px]] || style="text-align:left; background:#d3d3d3;"| [[File:Ludwig van Beethoven - Symphonie 5 c-moll - 1. Allegro con brio.ogg|110px]]|| style="text-align:left; background:silver;"|[[File:Wagner - die walkure fantasie.ogg|110px]] || style="text-align:left; background:darkGrey;"| [[File:Kraftwerk - computerwelt - computerwelt 2.ogg|110px]] || style="text-align:left; background:gray;"| [[File:Brother Louie MT.ogg|110px]]|| style="text-align:left; background:dimGrey;"| [[File:Rammstein - Ohne dich (sample).ogg|110px]]
|}
===Media===
[[File:FrankfurterBuchmesse2008.JPG|thumb|The [[Frankfurt Book Fair]] in 2008]]
Germany's television market is the largest in Europe, with some 34 million TV households.
The many regional and national public broadcasters are organised in line with the federal political structure. Around 90% of German households have cable or satellite TV, and viewers can choose from a variety of free-to-view public and commercial channels. Pay-TV services have not become popular or successful while public TV broadcasters [[ZDF]] and [[ARD (broadcaster)|ARD]] offer a range of digital-only channels.[http://news.bbc.co.uk/2/hi/europe/country_profiles/1047864.stm Country profile: Germany], BBC News. Retrieved 7 December 2007.
Germany is home to some of the world's largest media conglomerates, including [[Bertelsmann]], the [[Axel Springer AG]] and [[ProSiebenSat.1 Media]].
The German-speaking book publishers produce about 700 million copies of books every year, with about 80,000 titles, nearly 60,000 of them new publications. Germany is in third place on international statistics after the English-speaking book market and the People’s Republic of China.[http://www.land-der-ideen.de/CDA/facts_printing,4563,0,,en.html Germany – Land of ideas], www.land-der-ideen.de Retrieved 12 December 2010. The [[Frankfurt Book Fair]] is considered to be the most important book fair in the world for international deals and trading and has a tradition that spans over 500 years.
The country's news is provided for English speakers e.g. by news magazine [[Der Spiegel]] and state broadcaster [[Deutsche Welle]].
===Sports===
{{Main|Sport in Germany}}
[[File:Michael Schumacher 2010 Malaysia.jpg|thumb|upright|140px|[[Michael Schumacher]] has won seven [[Formula One]] championships.]]
Sport forms an integral part of German life. Twenty-seven million Germans are members of a sports club and an additional twelve million pursue such an activity individually.{{Cite web | url = http://www.germany.info/relaunch/culture/life/sports.html | title = Germany Info: Culture & Life: Sports | publisher=Germany Embassy in Washington, D.C | accessdate = 28 December 2006}} [[Association football]] is the most popular sport. With more than 6.3 million official members, the [[German Football Association]] (''Deutscher Fußball-Bund'') is the largest sports organisation of its kind worldwide. The [[Fußball-Bundesliga|Bundesliga]] attracts the second highest [[List of attendance figures at domestic professional sports leagues|average attendance]] of any professional sports league in the world. The [[Germany national football team|German national football team]] won the [[FIFA World Cup]] in 1954, 1974 and 1990 and the [[UEFA European Football Championship]] in 1972, 1980 and 1996. Germany has hosted the FIFA World Cup in [[FIFA World Cup 1974|1974]] and [[FIFA World Cup 2006|2006]] and the [[UEFA European Football Championship]] in [[UEFA Euro 1988|1988]]. Among the most successful and renowned footballers are
[[Franz Beckenbauer]], [[Gerd Müller]], [[Jürgen Klinsmann]], [[Lothar Matthäus]], and [[Oliver Kahn]]. Other popular spectator sports include [[team handball|handball]], volleyball, basketball, [[ice hockey]], and tennis.
[[File:AllianzArenaSunset.jpg|thumb|left|The [[Allianz Arena]] is host to the football club [[Bayern Munich]] and was a venue for the [[2006 FIFA World Cup]].]]
Germany is one of the leading motor sports countries in the world. Race-winning cars, teams and drivers have come from Germany. The most successful [[Formula One]] driver in history, [[Michael Schumacher]], has set many significant motor sport records during his career, having won more [[List of Formula One World Drivers' Champions|Formula One World Drivers' Championships]] and more Formula One races than any other driver since Formula One's debut season in 1950. He is one of the highest paid sportsmen in history and became a billionaire athlete.{{Cite news | url = http://sport.guardian.co.uk/formulaone/story/0,,1929091,00.html | title = What we will miss about Michael Schumacher | newspaper=The Guardian | date = 23 October 2006 | accessdate = 19 October 2007 | location=London | first=David | last=Ornstein }} Constructors like [[BMW]] and [[Mercedes-Benz|Mercedes]] are among the leading manufacturers in motor sport. Additionally, [[Porsche]] has won the [[24 Hours of Le Mans]], a prestigious annual endurance race held in France, 16 times, and [[Audi]] has won it 9 times. The [[Deutsche Tourenwagen Masters]] is a popular series in Germany.
Historically, German sportsmen have been some of the most successful contenders in the Olympic Games, ranking third in an [[all-time Olympic Games medal count]], combining East and West German medals. In the [[2008 Summer Olympics]], Germany finished fifth in the medal count,{{Cite web | url = http://results.beijing2008.cn/WRM/ENG/INF/GL/95A/GL0000000.shtml | title = Beijing 2008 Medal Table | publisher=The Official Website of the Beijing 2008 Olympic Games | accessdate = 8 September 2008}} while in the [[2006 Winter Olympics]] they finished first.{{Cite web | url = http://www.olympic.org/uk/games/past/table_uk.asp?OLGT=2&OLGY=2006 | title = Turin 2006 Medal Table | publisher=International Olympic Committee | accessdate = 28 December 2006}}
Germany has hosted the Summer Olympic Games twice, in Berlin in [[1936 Summer Olympics|1936]] and in [[Munich]] in [[1972 Summer Olympics|1972]]. The Winter Olympic Games took place in Germany once in [[1936 Winter Olympics|1936]] when they were staged in the [[Bavaria]]n twin towns of [[Garmisch]] and [[Partenkirchen]].
===Cuisine===
{{Main|German cuisine}}
[[File:144215094 be5e39f10f b.jpg|thumb|A ''[[Schwarzwälder Kirschtorte]]'' (Black Forest cake)]]
German cuisine varies from region to region. The southern regions of [[Bavarian cuisine|Bavaria]] and Swabia, for instance, share a culinary culture with Switzerland and Austria. Pork, beef, and poultry are the main varieties of meat consumed in Germany, with pork being the most popular.{{Cite web | url = http://www.nationmaster.com/country/gm-germany/foo-food | title = German food stats | publisher=www.nationmaster.com | accessdate = 26 November 2007}} Throughout all regions, meat is often eaten in sausage form. More than 1500 different types of [[sausage]] are produced in Germany. [[Organic food]] has gained a market share of around 3.0%, and is predicted to increase further.{{Cite web | url = http://www.organic-europe.net/country_reports/germany/default.asp | title = Organic Agriculture in Germany | publisher=organic-Europe | accessdate = 26 November 2007}}
A popular German saying has the meaning: "Breakfast like an emperor, lunch like a king, and dine like a beggar." Breakfast is usually a selection of breads and rolls with jam and honey or cold meats and cheese, sometimes accompanied by a boiled egg. Cereals or muesli with milk or [[yoghurt]] is less common but widespread.[http://www.cp-pc.ca/english/germany/eating.html Eating the German way], Cultural Profiles Project. Retrieved 2007, 26 November. More than 300 types of bread are sold in bakery shops across the country.[http://www.germany-tourism.de/ENG/culture_and_events/culinary_germany_brot.htm 300 Types of Bread], www.germany-tourism.de. Retrieved 2007, 26 November.
[[File:Buffet Germany.jpg|thumb|left|A typical cheese and cold meat [[buffet]] served at private festivities]]
As a country with many immigrants, Germany has adopted many international dishes into its cuisine and daily eating habits. [[Italian cuisine|Italian]] dishes like pizza and pasta, [[Turkish cuisine|Turkish]] and [[Arab]] dishes like [[Doner kebab|Döner Kebab]] and [[Falafel]] are well established, especially in bigger cities. International [[Hamburger|burger]] chains, as well as [[Chinese cuisine|Chinese]] and [[Greek cuisine|Greek]] restaurants, are widespread. [[Indian cuisine|Indian]], [[Thai cuisine|Thai]], [[Japanese cuisine|Japanese]], and other Asian cuisines have gained popularity in recent decades.
Among high-profile restaurants in Germany, the [[Michelin guide]] has awarded nine restaurants three stars, the highest designation, while 15 more received two stars.[http://www.dw-world.de/dw/article/0,2144,2914502,00.html Schnitzel Outcooks Spaghetti in Michelin Guide], Deutsche Welle. Retrieved 2007, 26 November. German restaurants have become the world's second most decorated eateries after France.[http://in.reuters.com/article/lifestyleMolt/idINL1447732320071114 German cuisine beats Italy, Spain in gourmet stars], Reuters India. Retrieved 2007, 26 November.
Although wine is becoming more popular in many parts of Germany, the national alcoholic drink is beer. German beer consumption per person is declining but—at 116 litres annually—it is still among the highest in the world.[http://www.royalunibrew.com/Default.aspx?ID=266 Europe's largest beer market], royalunibrew.com. Retrieved 2007, 26 November. Beer varieties include [[Altbier|Alt]], [[Bock]], [[Dunkel]], [[Kölsch (beer)|Kölsch]], [[Lager]], [[Malzbier]], [[Pilsener|Pils]], and [[Weizenbier]]. Among 18 surveyed western countries, Germany ranked 14th in the list of per capita consumption of soft drinks in general, while it ranked third in the consumption of fruit juices.[http://www.nationmaster.com/graph/foo_sof_dri_con-food-soft-drink-consumption Soft drink consumption], www.nationmaster.com. Retrieved 2007, 26 November. Furthermore, carbonated mineral water and ''[[Spritzer#Non-alcoholic spritzer|Schorle]]'' (its mixture with fruit juice) are very popular in Germany.
===Society===
[[File:Schloss-Neuschwanstein.jpg|thumb|right|Tourism is an important industry in parts of Germany([[Neuschwanstein Castle]])]]
Germany is a modern, cosmopolitan, creative society, shaped by a plurality of lifestyles and regional identities.[http://www.germany.info/Vertretung/usa/en/07__Culture__Lifestyle__Travel/01/__Society.html Society] The German Mission to the United states. Retrieved 16 October 2010. The country has established a high level of [[gender equality]], promotes [[Disability rights movement|disability rights]], and is legally and socially tolerant towards homosexuals. Gays and lesbians can legally adopt their partner's biological children, and [[civil union]]s have been permitted since 2001.[http://www.news24.com/News24/World/News/0,,2-10-1462_1613010,00.html Germany extends gay rights]{{dead link|date=September 2010}} News24.com. Retrieved 25 November 2007. The Vice-Chancellor [[Guido Westerwelle]] and the mayor of Berlin, [[Klaus Wowereit]], are openly gay.{{Cite news |last=Weinthal |first=Benjamin |title=He's Gay, and That's Okay |newspaper=[[Gay City News]] | location = New York |date=31 August 2006 |url=http://www.gaycitynews.com/articles/2006/08/31/gay_city_news_archives/past%20issues/17334472.txt |accessdate=3 September 2009}}{{Update after|2011|09|reason=elections in Berlin}}
During the last decade of the 20th century, Germany changed its attitude towards immigrants. Until the mid-1990s the opinion was widespread that Germany is not a country of immigration, even though about 20% of the population were of non-German origin. Today the government and a majority of the German society are acknowledging that immigrants from diverse ethnocultural backgrounds are part of the German society and that controlled immigration should be initiated based on qualification standards.{{Cite book | first = Friedrich | last = Heckmann | url = http://books.google.com/?id=l9yTtCAnQWwC&pg=PA53&lpg=PA53&dq=cdu+immigrant+einwanderungsland#PPA53,M1 | title = The Integration of Immigrants in European Societies: national differences and trends of convergence| publisher=Lucius & Lucius | location = Stuttgart | pages = 51 ff. | year = 2003 | isbn = 978-3-8282-0181-1 | accessdate = 28 October 2010 }}
Since the 2006 FIFA World Cup, the internal and external evaluation of Germany's national image has changed.[http://www.brandovation.net/ How Germany won the World Cup of Nation Branding] BrandOvation. Retrieved 25 November 2007. In annually conducted global surveys known as [[Nation Brands Index]], Germany became significantly and repeatedly more highly ranked after the tournament. People in 20 different states assessed the country's reputation in terms of culture, politics, exports, its people and its attractiveness to tourists, immigrants and investments. Germany has been named the world's second most valued nation among 50 countries in 2010.{{Cite press release | title = 2010 Anholt-GfK Roper Nation Brands Index | publisher=[[GfK]] | date = 12 October 2010 | url = http://www.gfk.com/group/press_information/press_releases/006688/index.en.html | accessdate = 15 October 2010 }} Another global opinion poll, for the [[BBC]], revealed that Germany is recognised for the most positive influence in the world in 2010. A majority of 59% have a positive view of the country, while 14% have a negative view.{{Cite news | url = http://news.bbc.co.uk/2/hi/8626041.stm | title = World warming to US under Obama, BBC poll suggests |publisher=BBC News Online | location = London | date = 19 April 2010 |accessdate = 28 October 2010 }}[http://news.bbc.co.uk/2/shared/bsp/hi/pdfs/160410bbcwspoll.pdf BBC World Service Poll], BBC News. Retrieved 19 April 2010.
With an expenditure of €67 billion on international travel in 2008, Germans spent more money on travel than any other country. The most visited destinations were Spain, Italy and Austria.{{Cite news | url = http://economictimes.indiatimes.com/ET-Cetera/Germans-spend-most-on-foreign-trips-Industry-group/articleshow/4250332.cms | title = Germans spend most on foreign trips: Industry group] | date = 10 March 2009 | location = New Delhi | newspaper=The Economic Times |accessdate = 15 March 2009 }}
{{-}}
==See also==
{{Portal box|Germany|European Union}}
* [[Outline of Germany]]
{{Template group
|title = Articles Related to Germany
|list =
{{Germany topics}}
{{States of Germany |titlestyle=background:#ddf;}}
{{Template group
|title = Geographic locale
|list =
{{Geographic location
| Northwest = ''[[North Sea]]''
{{Flag|The Netherlands}}
| North = {{Flag|Denmark}}
| Northeast = ''[[Baltic Sea]]''
| West = {{Flag|Belgium}}
{{Flag|Luxembourg}}
| Centre = {{Flagicon|Germany}} [[Outline of Germany|Germany]]
| East = {{Flag|Poland}}
{{Flag|Czech Republic}}
| Southwest = {{Flag|France}}
| South = {{Flag|Switzerland}} • {{Flag|Austria}}
| Southeast = {{Flag|Austria}}
}}
{{Sovereign states of Europe}}
{{Countries bordering the Baltic Sea}}
}}
{{Template group
|title = International membership
|list =
{{EU members}}
{{North Atlantic Treaty Organisation (NATO)}}
{{Organisation for Economic Co-operation and Development}}
{{G8 nations}}
{{G20}}
{{WTO members}}
{{Council of Europe}}
{{OSCE}}
}}
{{National personifications}}
}}
==Notes==
{{Reflist|group=note}}
==References==
{{Reflist|colwidth=30em}}
==External links==
{{Spoken Wikipedia-2|2008-06-24|Germany (Spoken Article) Part 1.ogg|Germany (Spoken Article) Part 2.ogg}}
{{Sister project links|Germany}}
* [http://www.deutschland.de/ Deutschland.de] – Official Germany portal (non-profit)
* [http://www.bundeskanzlerin.de/ Official site of German Chancellor] (German and English)
* [http://www.dw-world.de/ Deutsche Welle] – Germany's international broadcaster
* [https://www.cia.gov/library/publications/world-leaders-1/world-leaders-g/germany.html Chief of State and Cabinet Members]
* {{CIA World Factbook link|gm|Germany}}
* [http://www.britannica.com/EBchecked/topic/231186/Germany Germany] entry at ''[[Encyclopædia Britannica]]''
* [http://ucblibraries.colorado.edu/govpubs/for/germany.htm Germany] at ''UCB Libraries GovPubs''
* {{dmoz|Regional/Europe/Germany}}
{{Wikiatlas|Germany}}
* [http://www.tatsachen-ueber-deutschland.de/en/ Facts about Germany] – by the German Federal Foreign Office
* [http://www.destatis.de/e_home.htm Destatis.de] – Federal Statistical Office Germany {{en icon}}
{{osmrelation-inline|51477}}
[[Category:Germany| ]]
[[Category:European countries]]
[[Category:Member states of the European Union]]
[[Category:Alpine countries]]
[[Category:Countries bordering the Atlantic Ocean]]
[[Category:Countries bordering the Baltic Sea]]
[[Category:Republics]]
[[Category:Federal countries]]
[[Category:German-speaking countries]]
[[Category:G8 nations]]
[[Category:G20 nations]]
[[Category:Liberal democracies]]
[[Category:Western Europe]]
[[Category:Member states of the Union for the Mediterranean]]
[[Category:States and territories established in 1871]]
[[Category:Members of the North Atlantic Treaty Organization]]
{{Link GA|da}}
{{Link GA|de}}
{{Link GA|eo}}
{{Link GA|es}}
{{Link GA|lt}}
{{Link GA|zh}}
{{Link FA|af}}
{{Link FA|gv}}
{{Link FA|hr}}
{{Link FA|lv}}
{{Link FA|mr}}
{{Link FA|nah}}
{{Link FA|pt}}
{{Link FA|sq}}
{{Link FA|tr}}
{{Link FA|yi}}
[[ace:Jeureuman]]
[[af:Duitsland]]
[[als:Deutschland]]
[[am:ጀርመን]]
[[ang:Þēodscland]]
[[ab:Алмантәыла]]
[[ar:ألمانيا]]
[[an:Alemanya]]
[[arc:ܓܪܡܢ]]
[[roa-rup:Ghermãnia]]
[[frp:Alemagne]]
[[ast:Alemaña]]
[[gn:Alemaña]]
[[ay:Alimaña]]
[[az:Almaniya]]
[[bm:Jermani]]
[[bn:জার্মানি]]
[[zh-min-nan:Tek-kok]]
[[ba:Германия]]
[[be:Германія]]
[[be-x-old:Нямеччына]]
[[bcl:Alemanya]]
[[bi:Germany]]
[[bar:Deitschland]]
[[bo:འཇར་མན།]]
[[bs:Njemačka]]
[[br:Alamagn]]
[[bg:Германия]]
[[ca:Alemanya]]
[[cv:Германи]]
[[ceb:Alemanya]]
[[cs:Německo]]
[[cbk-zam:Alemania]]
[[tum:Germany]]
[[co:Germania]]
[[cy:Yr Almaen]]
[[da:Tyskland]]
[[pdc:Deitschland]]
[[de:Deutschland]]
[[dv:ޖަރުމަނުވިލާތް]]
[[nv:Béésh Bichʼahii Bikéyah]]
[[dsb:Nimska]]
[[dz:ཇཱར་མ་ནི་]]
[[et:Saksamaa]]
[[el:Γερμανία]]
[[eml:Germâgna]]
[[myv:Германия Мастор]]
[[es:Alemania]]
[[eo:Germanio]]
[[ext:Alemaña]]
[[eu:Alemania]]
[[ee:Germany]]
[[fa:آلمان]]
[[hif:Germany]]
[[fo:Týskland]]
[[fr:Allemagne]]
[[fy:Dútslân]]
[[fur:Gjermanie]]
[[ga:An Ghearmáin]]
[[gv:Yn Ghermaan]]
[[gd:A' Ghearmailt]]
[[gl:Alemaña - Deutschland]]
[[frr:Tjüschlönj]]
[[gan:德國]]
[[gag:Germaniya]]
[[got:𐌸𐌹𐌿𐌳𐌹𐍃𐌺𐌰𐌻𐌰𐌽𐌳]]
[[hak:Tet-koet]]
[[xal:Ниицәтә Немшин Орн]]
[[ko:독일]]
[[haw:Kelemānia]]
[[hy:Գերմանիա]]
[[hi:जर्मनी]]
[[hsb:Němska]]
[[hr:Njemačka]]
[[io:Germania]]
[[ig:Jémanị]]
[[ilo:Alemania]]
[[bpy:জার্মানি]]
[[id:Jerman]]
[[ia:Germania]]
[[ie:Germania]]
[[iu:ᔮᒪᓂ/jaamani]]
[[os:Герман]]
[[xh:IJamani]]
[[zu:IJalimani]]
[[is:Þýskaland]]
[[it:Germania]]
[[he:גרמניה]]
[[jv:Jerman]]
[[kl:Tysklandi]]
[[kn:ಜರ್ಮನಿ]]
[[pam:Alemania]]
[[ka:გერმანია]]
[[ks:जर्मनी]]
[[csb:Miemieckô]]
[[kk:Алмания]]
[[kw:Almayn]]
[[rw:Ubudage]]
[[ky:Германия]]
[[rn:Ubudagi]]
[[sw:Ujerumani]]
[[kv:Германия]]
[[kg:Alemanyi]]
[[ht:Almay]]
[[ku:Almanya]]
[[koi:Немечму]]
[[lad:Almania]]
[[krc:Германия]]
[[lo:ປະເທດເຢັຽລະມັນ]]
[[la:Germania]]
[[lv:Vācija]]
[[lb:Däitschland]]
[[lt:Vokietija]]
[[lij:Germania]]
[[li:Duutsjlandj]]
[[ln:Alémani]]
[[jbo:dotygu'e]]
[[lg:Girimane]]
[[lmo:Germania]]
[[hu:Németország]]
[[mk:Германија]]
[[mg:Alemaina]]
[[ml:ജർമ്മനി]]
[[mt:Ġermanja]]
[[mi:Tiamana]]
[[mr:जर्मनी]]
[[arz:المانيا]]
[[mzn:آلمان]]
[[ms:Jerman]]
[[cdo:Dáik-guók]]
[[mdf:Германие мастор]]
[[mn:Герман]]
[[my:ဂျာမနီနိုင်ငံ]]
[[nah:Teutontlālpan]]
[[na:Germany]]
[[nl:Duitsland]]
[[nds-nl:Duutslaand]]
[[ne:जर्मनी]]
[[new:जर्मनी]]
[[ja:ドイツ]]
[[nap:Germania]]
[[ce:Германи]]
[[pih:Doichland]]
[[no:Tyskland]]
[[nn:Tyskland]]
[[nrm:Allemangne]]
[[nov:Germania]]
[[oc:Alemanha]]
[[mhr:Немыч Эл]]
[[uz:Olmoniya]]
[[pa:ਜਰਮਨੀ]]
[[pag:Germany]]
[[pnb:جرمنی]]
[[pap:Alemania]]
[[ps:آلمان]]
[[km:អាល្លឺម៉ង់]]
[[pcd:Alemanne]]
[[pfl:Daitschlond]]
[[pms:Gërmania]]
[[tpi:Siamani]]
[[nds:Düütschland]]
[[pl:Niemcy]]
[[pnt:Γερμανία]]
[[pt:Alemanha]]
[[kaa:Germaniya]]
[[crh:Almaniya]]
[[ty:Heremani]]
[[ksh:Dütschland]]
[[ro:Germania]]
[[rmy:Jermaniya]]
[[rm:Germania]]
[[qu:Alimanya]]
[[ru:Германия]]
[[rue:Нїмецько]]
[[sah:Германия]]
[[se:Duiska]]
[[sm:Siamani]]
[[sa:जर्मनी]]
[[sc:Germània]]
[[sco:Germany]]
[[stq:Düütsklound]]
[[sq:Gjermania]]
[[scn:Girmania]]
[[simple:Germany]]
[[ss:IJalimane]]
[[sk:Nemecko]]
[[cu:Нѣ́мьци]]
[[sl:Nemčija]]
[[szl:Mjymcy]]
[[so:Jarmalka]]
[[ckb:ئاڵمانیا]]
[[srn:Doysrikondre]]
[[sr:Немачка]]
[[sh:Nemačka]]
[[su:Jérman]]
[[fi:Saksa]]
[[sv:Tyskland]]
[[tl:Alemanya]]
[[ta:செருமனி]]
[[kab:Lalman]]
[[tt:Алмания]]
[[te:జర్మనీ]]
[[tet:Alemaña]]
[[th:ประเทศเยอรมนี]]
[[ti:ጀርመን]]
[[tg:Олмон]]
[[chr:ᎠᏛᏥ]]
[[tr:Almanya]]
[[tk:Germaniýa]]
[[udm:Германия]]
[[uk:Німеччина]]
[[ur:جرمنی]]
[[ug:گېرمانىيە]]
[[za:Dwzgoz]]
[[vec:Germania]]
[[vi:Đức]]
[[vo:Deutän]]
[[fiu-vro:S'aksamaa]]
[[wa:Almagne]]
[[zh-classical:德國]]
[[vls:Duutsland]]
[[war:Alemanya]]
[[wo:Almaañ]]
[[wuu:德国]]
[[ts:Jarimani]]
[[yi:דייטשלאנד]]
[[yo:Jẹ́mánì]]
[[zh-yue:德國]]
[[diq:Almanya]]
[[zea:Duutsland]]
[[bat-smg:Vuokītėjė]]
[[zh:德国]] sweble.wikitext 0000664 0000000 0000000 00000000507 13352705604 0042612 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization-eval/src/test/resources/various The '''Sweble''' Wikitext parser is an
[[Open-source software|open-source tool]]
to [[Parsing|parse]] the [[Wikitext]]
[[markup language]] used by [[MediaWiki]],
the software behind [[Wikipedia]].
== The current state of parsing ==
* list item 1
* list item 2
== How Sweble works ==
another section
[[Category:Parsing]] sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/ 0000775 0000000 0000000 00000000000 13352705604 0031417 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/pom.xml 0000664 0000000 0000000 00000011311 13352705604 0032731 0 ustar 00root root 0000000 0000000
4.0.0
org.sweble.engine
sweble-engine-parent
3.1.9
sweble-engine-serialization
jar
Sweble Engine Serialization
2013
A module for serializing and deserializing WOM as JSON or XML.
../
org.apache.commons
commons-compress
commons-io
commons-io
xerces
xercesImpl
com.google.code.gson
gson
org.sweble.wikitext
swc-parser-lazy
org.sweble.wikitext
swc-engine
org.sweble.wom3
sweble-wom3-core
org.sweble.wom3
sweble-wom3-json-tools
org.sweble.wom3
sweble-wom3-swc-adapter
com.mycila
license-maven-plugin
src/test/resources/**/*.wikitext
de.thetaphi
forbiddenapis
commons-io-unsafe-1.4
org.apache.maven.plugins
maven-dependency-plugin
xml-apis:xml-apis
sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/ 0000775 0000000 0000000 00000000000 13352705604 0032206 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/ 0000775 0000000 0000000 00000000000 13352705604 0033132 5 ustar 00root root 0000000 0000000 java/ 0000775 0000000 0000000 00000000000 13352705604 0033774 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main org/ 0000775 0000000 0000000 00000000000 13352705604 0034563 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java sweble/ 0000775 0000000 0000000 00000000000 13352705604 0036044 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org engine/ 0000775 0000000 0000000 00000000000 13352705604 0037311 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org/sweble serialization/ 0000775 0000000 0000000 00000000000 13352705604 0042166 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org/sweble/engine CompressionException.java 0000664 0000000 0000000 00000002261 13352705604 0047212 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
public class CompressionException
extends
Exception
{
private static final long serialVersionUID = 1L;
public CompressionException()
{
}
public CompressionException(String message, Throwable cause)
{
super(message, cause);
}
public CompressionException(String message)
{
super(message);
}
public CompressionException(Throwable cause)
{
super(cause);
}
}
CompressorFactory.java 0000664 0000000 0000000 00000006460 13352705604 0046523 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.commons.compress.compressors.CompressorException;
import org.apache.commons.compress.compressors.CompressorStreamFactory;
public class CompressorFactory
{
public static enum CompressionFormat
{
BZIP2
{
@Override
public OutputStream createCompressorOutputStream(OutputStream out) throws CompressorException
{
return (new CompressorStreamFactory()).createCompressorOutputStream(
CompressorStreamFactory.BZIP2, out);
}
@Override
public InputStream createCompressorInputStream(InputStream in) throws CompressorException
{
return (new CompressorStreamFactory()).createCompressorInputStream(
CompressorStreamFactory.BZIP2, in);
}
},
GZIP
{
@Override
public OutputStream createCompressorOutputStream(OutputStream out) throws CompressorException
{
return (new CompressorStreamFactory()).createCompressorOutputStream(
CompressorStreamFactory.GZIP, out);
}
@Override
public InputStream createCompressorInputStream(InputStream in) throws CompressorException
{
return (new CompressorStreamFactory()).createCompressorInputStream(
CompressorStreamFactory.GZIP, in);
}
},
XZ
{
@Override
public OutputStream createCompressorOutputStream(OutputStream out) throws CompressorException
{
return (new CompressorStreamFactory()).createCompressorOutputStream(
CompressorStreamFactory.XZ, out);
}
@Override
public InputStream createCompressorInputStream(InputStream in) throws CompressorException
{
return (new CompressorStreamFactory()).createCompressorInputStream(
CompressorStreamFactory.XZ, in);
}
};
public abstract OutputStream createCompressorOutputStream(
OutputStream out) throws CompressorException;
public abstract InputStream createCompressorInputStream(
InputStream in) throws CompressorException;
}
public static OutputStream createCompressorOutputStream(
CompressionFormat compressionFormat,
OutputStream out) throws CompressorFactoryException
{
try
{
return compressionFormat.createCompressorOutputStream(out);
}
catch (Exception e)
{
throw new CompressorFactoryException(e);
}
}
public static InputStream createCompressorInputStream(
CompressionFormat compressionFormat,
InputStream in) throws CompressorFactoryException
{
try
{
return compressionFormat.createCompressorInputStream(in);
}
catch (Exception e)
{
throw new CompressorFactoryException(e);
}
}
}
CompressorFactoryException.java 0000664 0000000 0000000 00000002317 13352705604 0050377 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
public class CompressorFactoryException
extends
Exception
{
private static final long serialVersionUID = 1L;
public CompressorFactoryException()
{
}
public CompressorFactoryException(String message, Throwable cause)
{
super(message, cause);
}
public CompressorFactoryException(String message)
{
super(message);
}
public CompressorFactoryException(Throwable cause)
{
super(cause);
}
}
DeserializationException.java 0000664 0000000 0000000 00000002305 13352705604 0050036 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
public class DeserializationException
extends
Exception
{
private static final long serialVersionUID = 1L;
public DeserializationException()
{
}
public DeserializationException(String message, Throwable cause)
{
super(message, cause);
}
public DeserializationException(String message)
{
super(message);
}
public DeserializationException(Throwable cause)
{
super(cause);
}
}
SerializationException.java 0000664 0000000 0000000 00000002273 13352705604 0047531 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
public class SerializationException
extends
Exception
{
private static final long serialVersionUID = 1L;
public SerializationException()
{
}
public SerializationException(String message, Throwable cause)
{
super(message, cause);
}
public SerializationException(String message)
{
super(message);
}
public SerializationException(Throwable cause)
{
super(cause);
}
}
SerializationLabToolbox.java 0000664 0000000 0000000 00000004255 13352705604 0047642 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
import org.sweble.wom3.Wom3Node;
import org.sweble.wom3.serialization.Wom3JsonTypeAdapterInterface;
import org.sweble.wom3.serialization.Wom3NodeCompactJsonTypeAdapter;
import org.sweble.wom3.serialization.Wom3NodeJsonTypeAdapter;
import org.w3c.dom.Node;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
public class SerializationLabToolbox
{
public static Wom3JsonTypeAdapterInterface createWom3JsonTypeAdapter(
boolean compact)
{
return (compact ?
(new Wom3NodeCompactJsonTypeAdapter()) :
(new Wom3NodeJsonTypeAdapter()));
}
public static GsonBuilder registerWom3GsonAdapter(
GsonBuilder builder,
boolean compact)
{
Wom3JsonTypeAdapterInterface converter = createWom3JsonTypeAdapter(compact);
builder.registerTypeHierarchyAdapter(Node.class, converter);
builder.serializeNulls();
return builder;
}
public static Gson createWom3Gson(boolean compact, boolean pretty)
{
GsonBuilder builder = registerWom3GsonAdapter(new GsonBuilder(), compact);
if (pretty)
builder.setPrettyPrinting();
return builder.create();
}
public static String womToJson(
Wom3Node what,
boolean compact,
boolean pretty)
{
return createWom3Gson(compact, pretty).toJson(what);
}
public static Wom3Node jsonToWom(
String json,
boolean compact,
boolean pretty)
{
return createWom3Gson(compact, pretty).fromJson(json, Wom3Node.class);
}
}
WmToWomJsonApp.java 0000664 0000000 0000000 00000005266 13352705604 0045706 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
import java.io.File;
import java.io.IOException;
import java.nio.charset.Charset;
import org.sweble.wikitext.engine.EngineException;
import org.sweble.wikitext.parser.parser.LinkTargetException;
import org.sweble.wom3.serialization.Wom3NodeCompactJsonTypeAdapter;
import org.sweble.wom3.serialization.Wom3NodeJsonTypeAdapter;
import org.sweble.wom3.swcadapter.utils.WtWom3Toolbox;
import org.sweble.wom3.swcadapter.utils.WtWom3Toolbox.Artifacts;
import org.w3c.dom.Node;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonSerializer;
public class WmToWomJsonApp
extends
SerializationLabToolbox
{
public static void main(String[] args) throws LinkTargetException, IOException, EngineException
{
new WmToWomJsonApp().run(args);
}
private void run(String[] args) throws LinkTargetException, IOException, EngineException
{
if (args.length < 1)
{
System.err.println("Usage: java -cp JAR-with-dependencies.jar " +
getClass().getName() +
" WIKITEXT-FILE [compact|pretty]");
return;
}
boolean pretty = false;
boolean compact = false;
for (int i = 1; i < args.length; ++i)
{
if (args[i].compareToIgnoreCase("pretty") == 0)
pretty = true;
else if (args[i].compareToIgnoreCase("compact") == 0)
compact = true;
else
System.err.println("Unknown command line option: '" + args[i] + "'");
}
WtWom3Toolbox wtWomToolbox = new WtWom3Toolbox();
Artifacts af = wtWomToolbox.wmToWom(new File(args[0]), Charset.defaultCharset().name());
JsonSerializer converter =
compact ?
new Wom3NodeCompactJsonTypeAdapter() :
new Wom3NodeJsonTypeAdapter();
GsonBuilder builder = new GsonBuilder();
builder.registerTypeHierarchyAdapter(Node.class, converter);
builder.serializeNulls();
if (pretty)
builder.setPrettyPrinting();
Gson gson = builder.create();
gson.toJson(af.womDoc, System.out);
}
}
WmToWomXmlApp.java 0000664 0000000 0000000 00000005657 13352705604 0045541 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.Charset;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
import org.sweble.wikitext.engine.EngineException;
import org.sweble.wikitext.parser.parser.LinkTargetException;
import org.sweble.wom3.swcadapter.utils.WtWom3Toolbox;
import org.sweble.wom3.swcadapter.utils.WtWom3Toolbox.Artifacts;
public class WmToWomXmlApp
extends
SerializationLabToolbox
{
public static void main(String[] args) throws TransformerException, LinkTargetException, IOException, EngineException
{
new WmToWomXmlApp().run(args);
}
private void run(String[] args) throws TransformerException, LinkTargetException, IOException, EngineException
{
if (args.length < 1)
{
System.err.println("Usage: java -cp JAR-with-dependencies.jar " +
getClass().getName() +
" WIKITEXT-FILE [pretty]");
return;
}
boolean pretty = false;
for (int i = 1; i < args.length; ++i)
{
if (args[i].compareToIgnoreCase("pretty") == 0)
pretty = true;
else
System.err.println("Unknown command line option: '" + args[i] + "'");
}
WtWom3Toolbox wtWomToolbox = new WtWom3Toolbox();
Artifacts af = wtWomToolbox.wmToWom(new File(args[0]), Charset.defaultCharset().name());
Transformer transformer;
if (pretty)
{
TransformerFactory tf = TransformerFactory.newInstance(
"org.apache.xalan.processor.TransformerFactoryImpl",
null);
InputStream xslt = getClass().getResourceAsStream("/org/sweble/wom3/pretty-print.xslt");
Transformer prettyXmlTransformer = tf.newTransformer(new StreamSource(xslt));
transformer = prettyXmlTransformer;
}
else
{
TransformerFactory tf = TransformerFactory.newInstance();
Transformer normalXmlTransformer = tf.newTransformer();
transformer = normalXmlTransformer;
}
transformer.transform(
new DOMSource(af.womDoc),
new StreamResult(System.out));
}
}
WomSerializer.java 0000664 0000000 0000000 00000031664 13352705604 0045637 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-engine-parent/sweble-engine-serialization/src/main/java/org/sweble/engine/serialization /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see http://www.gnu.org/licenses/.
*/
package org.sweble.engine.serialization;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.nio.charset.Charset;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerConfigurationException;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
import org.apache.commons.io.IOUtils;
import org.apache.xerces.parsers.DOMParser;
import org.sweble.engine.serialization.CompressorFactory.CompressionFormat;
import org.sweble.wom3.serialization.Wom3JsonTypeAdapterBase;
import org.sweble.wom3.serialization.Wom3NodeCompactJsonTypeAdapter;
import org.sweble.wom3.serialization.Wom3NodeJsonTypeAdapter;
import org.w3c.dom.Document;
import org.w3c.dom.DocumentFragment;
import org.w3c.dom.Node;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
import org.xml.sax.SAXNotRecognizedException;
import org.xml.sax.SAXNotSupportedException;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
public class WomSerializer
{
private static final Charset CHARSET = Charset.forName("UTF8");
// =========================================================================
public static enum SerializationFormat
{
JAVA,
JSON,
XML
}
// =========================================================================
private Transformer prettyXmlTransformer;
private Transformer normalXmlTransformer;
private String documentImplClassName = org.sweble.wom3.impl.DocumentImpl.class.getName();
// =========================================================================
public WomSerializer()
{
}
// =========================================================================
public String getDocumentImplClassName()
{
return documentImplClassName;
}
public void setDocumentImplClassName(String documentImplClassName)
{
this.documentImplClassName = documentImplClassName;
}
// =========================================================================
public byte[] serialize(
Document wom,
SerializationFormat serializationFormat,
boolean compact,
boolean pretty) throws IOException, SerializationException
{
final byte[] result;
switch (serializationFormat)
{
case JAVA:
{
ByteArrayOutputStream baos = null;
ObjectOutputStream oos = null;
try
{
baos = new ByteArrayOutputStream();
oos = new ObjectOutputStream(baos);
oos.writeObject(wom);
}
finally
{
IOUtils.closeQuietly(baos);
IOUtils.closeQuietly(oos);
}
result = baos.toByteArray();
break;
}
case JSON:
{
result = getGson(createDocumentForSerialization(), compact, pretty)
.toJson(wom)
.getBytes(CHARSET);
break;
}
case XML:
{
ByteArrayOutputStream baos = null;
try
{
baos = new ByteArrayOutputStream();
getXmlTransformer(pretty).transform(
new DOMSource(wom),
new StreamResult(baos));
}
catch (TransformerException e)
{
throw new SerializationException(e);
}
finally
{
IOUtils.closeQuietly(baos);
}
result = baos.toByteArray();
break;
}
default:
throw new IllegalArgumentException();
}
return result;
}
public Document deserialize(
byte[] serialized,
SerializationFormat serializationFormat,
boolean compact) throws IOException, DeserializationException
{
Document result;
switch (serializationFormat)
{
case JAVA:
{
ByteArrayInputStream bais = null;
ObjectInputStream ois = null;
try
{
bais = new ByteArrayInputStream(serialized);
ois = new ObjectInputStream(bais);
result = (Document) ois.readObject();
}
catch (ClassNotFoundException e)
{
throw new DeserializationException(e);
}
finally
{
IOUtils.closeQuietly(bais);
IOUtils.closeQuietly(ois);
}
break;
}
case JSON:
{
DocumentFragment fragment = (DocumentFragment)
getGson(createDocumentForDeserialization(), compact, false)
.fromJson(new String(serialized, CHARSET), Node.class);
Node firstChild = fragment.getFirstChild();
if (firstChild.getParentNode() != null)
firstChild.getParentNode().removeChild(firstChild);
Document doc = (Document) fragment.getOwnerDocument();
if (doc.getDocumentElement() != null)
doc.removeChild(doc.getDocumentElement());
doc.appendChild(firstChild);
result = doc;
break;
}
case XML:
{
ByteArrayInputStream bais = null;
try
{
bais = new ByteArrayInputStream(serialized);
InputSource is = new InputSource(bais);
DOMParser parser = getXmlParser();
parser.parse(is);
result = parser.getDocument();
}
catch (SAXException e)
{
throw new DeserializationException(e);
}
finally
{
IOUtils.closeQuietly(bais);
}
break;
}
default:
throw new IllegalArgumentException();
}
return result;
}
public byte[] compress(
byte[] serialized,
CompressionFormat compressionFormat) throws IOException, CompressionException
{
ByteArrayOutputStream out = null;
OutputStream cos = null;
InputStream in = null;
try
{
out = new ByteArrayOutputStream();
cos = CompressorFactory.createCompressorOutputStream(compressionFormat, out);
in = new ByteArrayInputStream(serialized);
IOUtils.copy(in, cos);
}
catch (CompressorFactoryException e)
{
throw new CompressionException(e);
}
finally
{
IOUtils.closeQuietly(in);
IOUtils.closeQuietly(cos);
IOUtils.closeQuietly(out);
}
return out.toByteArray();
}
public byte[] decompress(
byte[] compressed,
CompressionFormat compressionFormat) throws IOException, CompressionException
{
InputStream in = null;
InputStream cin = null;
ByteArrayOutputStream out = null;
try
{
in = new ByteArrayInputStream(compressed);
cin = CompressorFactory.createCompressorInputStream(compressionFormat, in);
out = new ByteArrayOutputStream(compressed.length * 2);
IOUtils.copy(cin, out);
}
catch (CompressorFactoryException e)
{
throw new CompressionException(e);
}
finally
{
IOUtils.closeQuietly(out);
IOUtils.closeQuietly(cin);
IOUtils.closeQuietly(in);
}
return out.toByteArray();
}
public byte[] serializeAndCompress(
Document wom,
CompressionFormat compressionFormat,
SerializationFormat serializationFormat,
boolean compact,
boolean pretty) throws IOException, CompressionException, SerializationException
{
ByteArrayOutputStream out = null;
OutputStream cos = null;
try
{
out = new ByteArrayOutputStream();
cos = CompressorFactory.createCompressorOutputStream(compressionFormat, out);
switch (serializationFormat)
{
case JAVA:
{
ObjectOutputStream oos = null;
try
{
oos = new ObjectOutputStream(cos);
oos.writeObject(wom);
}
finally
{
IOUtils.closeQuietly(oos);
}
break;
}
case JSON:
{
OutputStreamWriter osw = null;
try
{
osw = new OutputStreamWriter(cos, CHARSET);
Gson gson = getGson(createDocumentForSerialization(), compact, pretty);
gson.toJson(wom, osw);
}
finally
{
IOUtils.closeQuietly(osw);
}
break;
}
case XML:
{
getXmlTransformer(pretty).transform(
new DOMSource(wom),
new StreamResult(cos));
break;
}
default:
throw new IllegalArgumentException();
}
}
catch (CompressorFactoryException e)
{
throw new CompressionException(e);
}
catch (TransformerException e)
{
throw new SerializationException(e);
}
finally
{
IOUtils.closeQuietly(cos);
IOUtils.closeQuietly(out);
}
return out.toByteArray();
}
public Document decompressAndDeserialize(
byte[] compressed,
CompressionFormat compressionFormat,
SerializationFormat serializationFormat,
boolean compact) throws IOException, DeserializationException, CompressionException
{
ByteArrayInputStream in = null;
InputStream cin = null;
try
{
in = new ByteArrayInputStream(compressed);
cin = CompressorFactory.createCompressorInputStream(compressionFormat, in);
Document result;
switch (serializationFormat)
{
case JAVA:
{
ObjectInputStream ois = null;
try
{
ois = new ObjectInputStream(cin);
result = (Document) ois.readObject();
}
catch (ClassNotFoundException e)
{
throw new DeserializationException(e);
}
finally
{
IOUtils.closeQuietly(ois);
}
break;
}
case JSON:
{
InputStreamReader isr = null;
try
{
isr = new InputStreamReader(cin, CHARSET);
Gson gson = getGson(createDocumentForDeserialization(), compact, false);
result = gson.fromJson(isr, Document.class);
}
finally
{
IOUtils.closeQuietly(isr);
}
break;
}
case XML:
{
InputSource is = new InputSource(cin);
DOMParser parser = getXmlParser();
parser.parse(is);
result = parser.getDocument();
break;
}
default:
throw new IllegalArgumentException();
}
return result;
}
catch (CompressorFactoryException e)
{
throw new CompressionException(e);
}
catch (SAXException e)
{
throw new DeserializationException(e);
}
finally
{
IOUtils.closeQuietly(cin);
IOUtils.closeQuietly(in);
}
}
// =========================================================================
private Transformer getNormalXmlTransformer() throws TransformerConfigurationException
{
if (normalXmlTransformer == null)
{
TransformerFactory tf = TransformerFactory.newInstance();
normalXmlTransformer = tf.newTransformer();
}
return normalXmlTransformer;
}
private Transformer getPrettyXmlTransformer() throws TransformerConfigurationException
{
if (prettyXmlTransformer == null)
{
TransformerFactory tf = TransformerFactory.newInstance(
"net.sf.saxon.TransformerFactoryImpl",
null);
InputStream xslt = getClass().getResourceAsStream("/org/sweble/wom3/pretty-print.xslt");
prettyXmlTransformer = tf.newTransformer(new StreamSource(xslt));
}
return prettyXmlTransformer;
}
private Transformer getXmlTransformer(boolean pretty) throws TransformerConfigurationException
{
return pretty ?
getPrettyXmlTransformer() :
getNormalXmlTransformer();
}
private DOMParser getXmlParser() throws SAXNotRecognizedException, SAXNotSupportedException
{
DOMParser parser = new DOMParser();
parser.setProperty(
"http://apache.org/xml/properties/" + "dom/document-class-name",
documentImplClassName);
return parser;
}
// =========================================================================
private Gson getGson(Document doc, boolean compact, boolean pretty)
{
GsonBuilder builder = new GsonBuilder();
Wom3JsonTypeAdapterBase typeAdapter = compact ?
(new Wom3NodeCompactJsonTypeAdapter()) :
(new Wom3NodeJsonTypeAdapter());
typeAdapter.setDoc(doc);
builder.registerTypeHierarchyAdapter(Node.class, typeAdapter);
builder.serializeNulls();
if (pretty)
builder.setPrettyPrinting();
return builder.create();
}
private Document createDocumentForDeserialization() throws DeserializationException
{
try
{
return (Document) Class.forName(documentImplClassName).newInstance();
}
catch (InstantiationException e)
{
throw new DeserializationException(e);
}
catch (IllegalAccessException e)
{
throw new DeserializationException(e);
}
catch (ClassNotFoundException e)
{
throw new DeserializationException(e);
}
}
private Document createDocumentForSerialization() throws SerializationException
{
try
{
return (Document) Class.forName(documentImplClassName).newInstance();
}
catch (InstantiationException e)
{
throw new SerializationException(e);
}
catch (IllegalAccessException e)
{
throw new SerializationException(e);
}
catch (ClassNotFoundException e)
{
throw new SerializationException(e);
}
}
}
sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/ 0000775 0000000 0000000 00000000000 13352705604 0026606 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/LICENSE 0000664 0000000 0000000 00000026136 13352705604 0027623 0 ustar 00root root 0000000 0000000
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/pom.xml 0000664 0000000 0000000 00000016100 13352705604 0030121 0 ustar 00root root 0000000 0000000
4.0.0
org.sweble
sweble-parent
3.1.9
org.sweble.wikitext
sweble-wikitext-components-parent
pom
Sweble Wikitext Components Parent
2009
The Sweble Wikitext Components module provides a parser for MediaWiki's
wikitext and an engine trying to emulate the behavior of a MediaWiki.
swc-parser-lazy
swc-engine
swc-dumpreader
swc-article-cruncher
swc-example-basic
swc-example-dumpcruncher
swc-example-serialization
swc-example-xpath
${project.basedir}
${swc-parent.path}/../
0.8.0
1.4.3
1.3.0.1
4.3
1.15.0
1.4.8
net.sf.jopt-simple
jopt-simple
${jopt-simple.version}
xtc
rats-runtime
${rats-runtime.version}
com.thoughtworks.xstream
xstream
${xstream.version}
com.mycila
license-maven-plugin
JAVADOC_STYLE
JAVADOC_STYLE
SCRIPT_STYLE
org.jvnet.jaxb2.maven2
maven-jaxb2-plugin
${maven-jaxb2-plugin.version}
de.jflex
maven-jflex-plugin
${maven-jflex-plugin.version}
jflex-generate
generate-sources
generate
de.fau.cs.osr.maven.plugins
maven-rats-plugin
${maven-rats-plugin.version}
rats-generate
generate-sources
generate
org.eclipse.m2e
lifecycle-mapping
1.0.0
com.google.code.maven-replacer-plugin
maven-replacer-plugin
[1.0.0,)
replace
de.fau.cs.osr.maven.plugins
maven-rats-plugin
[1.0.0,)
generate
de.jflex
maven-jflex-plugin
[1.0.0,)
generate
Daniel Naber
naber@danielnaber.de
Developer
sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/ 0000775 0000000 0000000 00000000000 13352705604 0032632 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/pom.xml0000664 0000000 0000000 00000004766 13352705604 0034164 0 ustar 00root root 0000000 0000000
4.0.0
org.sweble.wikitext
sweble-wikitext-components-parent
3.1.9
swc-article-cruncher
jar
SWC - Sweble Article Cruncher
2012
A framework for multi-threaded processing of lots of articles.
../
org.slf4j
slf4j-api
de.fau.cs.osr.utils
utils
junit
junit
test
sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/ 0000775 0000000 0000000 00000000000 13352705604 0033421 5 ustar 00root root 0000000 0000000 main/ 0000775 0000000 0000000 00000000000 13352705604 0034266 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src java/ 0000775 0000000 0000000 00000000000 13352705604 0035207 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main org/ 0000775 0000000 0000000 00000000000 13352705604 0035776 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java sweble/ 0000775 0000000 0000000 00000000000 13352705604 0037257 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org wikitext/ 0000775 0000000 0000000 00000000000 13352705604 0041127 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble articlecruncher/ 0000775 0000000 0000000 00000000000 13352705604 0044304 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext Gatherer.java 0000664 0000000 0000000 00000004115 13352705604 0046711 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
import java.util.concurrent.BlockingQueue;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
public class Gatherer
extends
WorkerBase
{
private final BlockingQueue inTray;
private final BlockingQueue processedJobs;
private final BlockingQueue outTray;
// =========================================================================
public Gatherer(
AbortHandler abortHandler,
BlockingQueue inTray,
BlockingQueue processedJobs,
BlockingQueue outTray)
{
super(getClassName(), abortHandler);
Thread.currentThread().setName(getClassName());
this.inTray = inTray;
this.processedJobs = processedJobs;
this.outTray = outTray;
}
private static String getClassName()
{
return Gatherer.class.getSimpleName();
}
// =========================================================================
private int count = 0;
@Override
protected void work() throws Throwable
{
while (true)
{
Job processed = processedJobs.take();
++count;
processed.signOff(getClass(), null);
// TODO: Decide what to do with it.
boolean tryAgain = false;
if (tryAgain)
{
inTray.put(processed);
}
else
{
outTray.put(processed);
}
}
}
@Override
protected void after()
{
info(getClass().getSimpleName() + " counts " + count + " items");
}
}
Job.java 0000664 0000000 0000000 00000010467 13352705604 0045671 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
import java.util.List;
import org.sweble.wikitext.articlecruncher.JobTrace.Signer;
/**
* A job with a history and a trace.
*
* A job is one unit of work that is generated by a job generator (see also
* {@link JobGeneratorFactory}). It is then processed in a processing node (see
* also {@link ProcessingNodeFactory}) and finally its result is stored by a
* storer (see also {@link StorerFactory}).
*
* To generate a job first inherit from this class and add a payload to your job
* class. This payload is then processed by a processing node.
*
* A job starts with a trace (and a job ID stored in that trace), no result or
* exception and in the UNPROCESSED state.
*
* After processing a job can have a result if processing was successful or an
* exception can be associated with the job in case of failure. It will then be
* in the HAS_RESULT or FAILED state. It will further have a single history
* element in its history chain. The state of a job as well as its result or an
* exception are actually stored in that history element. The Job class offers
* delegating getters for convenience.
*
* Further history elements can be present if a job failed multiple times and
* was retried.
*/
public abstract class Job
{
private final JobTrace trace;
private JobHistory history;
// =========================================================================
public Job()
{
this.trace = new JobTrace();
}
public Job(JobTrace trace)
{
this.trace = trace;
}
// =========================================================================
public JobTrace getTrace()
{
return trace;
}
public long getJobId()
{
return trace.getJobId();
}
public List getSigners()
{
return trace.getSigners();
}
public void signOff(Class signer, String signature)
{
trace.signOff(signer, signature);
}
// =========================================================================
/**
* Retrieve the latest history element of the job. The history of a job
* forms a chain. The head element (the element returned by this getter)
* represents the latest processing attempt.
*
* @return Return the latest history element.
*/
public JobHistory getHistory()
{
return history;
}
/**
* Set the result of a successfully processed job.
*/
public synchronized void processed(Object result)
{
history = new JobHistory(history, result);
}
/**
* Set the exception of a job for which processing failed.
*
* Such a job can be re-submitted.
*/
public synchronized void failed(Exception exception)
{
history = new JobHistory(history, exception);
}
// =========================================================================
/**
* Returns the state of this job.
*/
public synchronized JobProcessingState getState()
{
return history != null ? history.getState() : JobProcessingState.UNPROCESSED;
}
/**
* Returns the result produced in the last processing run.
*
* @return The result produced in the last processing run or
* null if the last processing run failed or if the job
* has not yet been processed.
*/
public synchronized Object getResult()
{
return (getState() == JobProcessingState.HAS_RESULT) ? history.getResult() : null;
}
/**
* Returns the exception that occurred in the last processing run.
*
* @return The exception that occurred in the last processing run or
* null if that last processing run succeeded or if the
* job has not yet been processed.
*/
public synchronized Exception getException()
{
return (getState() == JobProcessingState.FAILED) ? history.getException() : null;
}
}
JobGeneratorFactory.java 0000664 0000000 0000000 00000002007 13352705604 0051057 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
import java.util.concurrent.BlockingQueue;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
public interface JobGeneratorFactory
{
WorkerBase create(
AbortHandler abortHandler,
BlockingQueue inTray,
JobTraceSet jobTraces);
}
JobHistory.java 0000664 0000000 0000000 00000005047 13352705604 0047251 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
public class JobHistory
{
private final JobHistory previous;
private final JobProcessingState state;
private final Object result;
private final Exception exception;
// =========================================================================
/**
* Construct a history element for a successful processing run.
*/
public JobHistory(JobHistory previous, Object result)
{
this.previous = previous;
this.state = JobProcessingState.HAS_RESULT;
this.result = result;
this.exception = null;
}
/**
* Construct a history element for a failed processing run.
*/
public JobHistory(JobHistory previous, Exception exception)
{
this.previous = previous;
this.state = JobProcessingState.FAILED;
this.result = null;
this.exception = exception;
}
// =========================================================================
/**
* Return the history of the processing run preceding this processing run.
*
* @return Returns the previous processing run or null if there
* was no previous processing run.
*/
public JobHistory getPrevious()
{
return previous;
}
/**
* Get the processing state of the processing run represented by this
* history element.
*/
public JobProcessingState getState()
{
return state;
}
/**
* Get the result of the processing run represented by this history element.
*
* @return Returns the result if the processing run was successful.
* Otherwise, if processing failed, null is returned.
*/
public Object getResult()
{
return result;
}
/**
* Get the exception of the processing run represented by this history
* element.
*
* @return Returns the exception if the processing run failed. Otherwise, if
* processing was successful, null is returned.
*/
public Exception getException()
{
return exception;
}
}
JobProcessingState.java 0000664 0000000 0000000 00000001561 13352705604 0050722 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
public enum JobProcessingState
{
/** Processing ended with a result. */
HAS_RESULT,
/** Processing failed. */
FAILED,
/** Not yet processed. */
UNPROCESSED,
}
JobTrace.java 0000664 0000000 0000000 00000005477 13352705604 0046655 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.atomic.AtomicLong;
public class JobTrace
{
public static final class Signer
{
private Class signer;
private String message;
public Signer(Class signer, String message)
{
super();
this.signer = signer;
this.message = message;
}
public Class getSigner()
{
return signer;
}
public String getMessage()
{
return message;
}
@Override
public String toString()
{
String str = signer.getSimpleName();
if (message != null)
str += " <" + message + ">";
return str;
}
}
// =========================================================================
private static AtomicLong jobIdCounter = new AtomicLong(0);
private final long jobId = jobIdCounter.incrementAndGet();
private List signers;
// =========================================================================
public void signOff(Class signer, String signature)
{
if (signers == null)
signers = new ArrayList();
signers.add(new Signer(signer, signature));
}
public long getJobId()
{
return jobId;
}
public List getSigners()
{
return signers;
}
// =========================================================================
@Override
public int hashCode()
{
final int prime = 31;
int result = 1;
result = prime * result + (int) (jobId ^ (jobId >>> 32));
return result;
}
@Override
public boolean equals(Object obj)
{
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
JobTrace other = (JobTrace) obj;
if (jobId != other.jobId)
return false;
return true;
}
// =========================================================================
@Override
public String toString()
{
StringBuilder b = new StringBuilder();
b.append(String.format("[ Job Trace: Job Id = %9d", jobId));
boolean first = true;
b.append("; Signed by = ");
for (Signer signer : signers)
{
if (!first)
b.append(", ");
b.append(signer.toString());
first = false;
}
b.append(" ]");
return b.toString();
}
}
JobTraceSet.java 0000664 0000000 0000000 00000002625 13352705604 0047321 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
public class JobTraceSet
{
private Set traces = new HashSet();
// =========================================================================
public synchronized void add(JobTrace trace)
{
traces.add(trace);
}
public synchronized boolean remove(JobTrace trace)
{
boolean removed = traces.remove(trace);
notifyAll();
return removed;
}
public synchronized Set getTraces()
{
return Collections.unmodifiableSet(traces);
}
public synchronized void waitForCompletion(int timeoutInSeconds) throws InterruptedException
{
while (!traces.isEmpty())
wait(timeoutInSeconds * 1000);
}
}
Nexus.java 0000664 0000000 0000000 00000021016 13352705604 0046251 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingDeque;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.ExecutorType;
import org.sweble.wikitext.articlecruncher.utils.MyExecutorService;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
import org.sweble.wikitext.articlecruncher.utils.WorkerLauncher;
import org.sweble.wikitext.articlecruncher.utils.WorkerSynchronizer;
public class Nexus
{
public static enum NexusState
{
INITIALIZED,
RUNNING,
SHUTDOWN
}
// =========================================================================
private static final Logger logger = LoggerFactory.getLogger(Nexus.class);
private static final int COMPLETION_TIMEOUT_IN_SECONDS = 60 * 5;
// =========================================================================
private BlockingQueue inTray;
private BlockingQueue processedJobs;
private BlockingQueue outTray;
private JobTraceSet jobTraces = new JobTraceSet();
private MyExecutorService executor;
private Throwable emergencyCause;
private WorkerLauncher gatherer;
private NexusState state;
private AbortHandler abortHandler;
private WorkerSynchronizer synchronizer = new WorkerSynchronizer();
private List jobGenerators = new ArrayList();
private List processingNodes = new ArrayList();
private List storers = new ArrayList();
// =========================================================================
public Nexus()
{
}
// =========================================================================
public void setUp(
int inTrayCapacity,
int processedJobsCapacity,
int outTrayCapacity) throws Throwable
{
synchronized (synchronizer.getMonitor())
{
if (state != null)
throw new IllegalStateException("Can only set-up Nexus once");
try
{
logger.info("Nexus starting");
inTray = new LinkedBlockingDeque(inTrayCapacity);
processedJobs = new LinkedBlockingDeque(processedJobsCapacity);
outTray = new LinkedBlockingDeque(outTrayCapacity);
executor = new MyExecutorService(ExecutorType.CACHED_THREAD_POOL, logger);
abortHandler = new AbortHandler()
{
@Override
public void notify(Throwable t)
{
emergencyShutdown(t);
}
};
gatherer = new WorkerLauncher(new WorkerInstantiator()
{
@Override
public WorkerBase instantiate()
{
return new Gatherer(abortHandler, inTray, processedJobs, outTray);
}
}, abortHandler);
state = NexusState.INITIALIZED;
}
catch (Throwable t)
{
logger.error("Nexus hit by exception", t);
setEmergencyCause(t);
nexusStopped();
}
}
}
public void start() throws Throwable
{
try
{
synchronized (synchronizer.getMonitor())
{
if (state != NexusState.INITIALIZED)
throw new IllegalStateException("Can only start Nexus after initialization");
state = NexusState.RUNNING;
try
{
logger.info("Nexus starting workers");
gatherer.start(executor);
logger.info("Nexus waiting for end of input stream");
synchronizer.waitForAll(1);
}
catch (InterruptedException e)
{
logger.error("Nexus interrupted unexpectedly", e);
setEmergencyCause(e);
}
catch (Throwable t)
{
logger.error("Nexus hit by exception", t);
setEmergencyCause(t);
}
}
if (emergencyCause == null)
{
logger.info("Nexus waiting for processing to finish");
jobTraces.waitForCompletion(COMPLETION_TIMEOUT_IN_SECONDS);
}
}
finally
{
MyExecutorService exec = null;
synchronized (synchronizer.getMonitor())
{
logger.info("Nexus shutting down");
exec = stopAll();
}
if (exec != null)
exec.shutdownAndAwaitTermination();
nexusStopped();
}
}
public void addJobGenerator(final JobGeneratorFactory factory)
{
synchronized (synchronizer.getMonitor())
{
switch (state)
{
case INITIALIZED:
case RUNNING:
{
logger.info("Adding job generator");
WorkerLauncher wl = new WorkerLauncher(new WorkerInstantiator()
{
@Override
public WorkerBase instantiate()
{
return factory.create(abortHandler, inTray, jobTraces);
}
}, abortHandler);
jobGenerators.add(wl);
wl.start(executor, synchronizer);
break;
}
default:
throw new IllegalStateException("Can only add job generators to initialized/running Nexus");
}
}
}
public void addProcessingNode(final ProcessingNodeFactory factory)
{
synchronized (synchronizer.getMonitor())
{
switch (state)
{
case INITIALIZED:
case RUNNING:
{
logger.info("Adding processing node");
WorkerLauncher wl = new WorkerLauncher(new WorkerInstantiator()
{
@Override
public WorkerBase instantiate()
{
return factory.create(
abortHandler,
inTray,
processedJobs);
}
}, abortHandler);
processingNodes.add(wl);
wl.start(executor);
break;
}
default:
throw new IllegalStateException("Can only add processing nodes to initialized/running Nexus");
}
}
}
public void addStorer(final StorerFactory factory)
{
synchronized (synchronizer.getMonitor())
{
switch (state)
{
case INITIALIZED:
case RUNNING:
{
logger.info("Adding storer");
WorkerLauncher wl = new WorkerLauncher(new WorkerInstantiator()
{
@Override
public WorkerBase instantiate()
{
return factory.create(abortHandler, jobTraces, outTray);
}
}, abortHandler);
storers.add(wl);
wl.start(executor);
break;
}
default:
throw new IllegalStateException("Can only add storers to initialized/running Nexus");
}
}
}
public Set getJobTraces()
{
return jobTraces.getTraces();
}
public BlockingQueue getInTray()
{
return inTray;
}
public BlockingQueue getOutTray()
{
return outTray;
}
public/*static*/void shutdown()
{
internalShutdown(null);
}
public/*static*/void emergencyShutdown(Throwable t)
{
internalShutdown(t);
}
// =========================================================================
private/*static*/void internalShutdown(Throwable t)
{
synchronized (/*get().*/synchronizer.getMonitor())
{
switch (/*get().*/state)
{
case RUNNING:
{
WorkerSynchronizer sync = /*get().*/synchronizer;
if (!sync.isSynchronized() && !sync.isAborted())
{
/*get().*/setEmergencyCause(t);
logger.info(t == null ?
"Nexus performing orderly shutdown" :
"Nexus performing emergency shutdown");
sync.abort();
}
break;
}
default:
throw new IllegalStateException("Can only shutdown running Nexus");
}
}
}
private void setEmergencyCause(Throwable t)
{
// Don't overwrite original cause
if (emergencyCause == null)
emergencyCause = t;
}
private MyExecutorService stopAll()
{
synchronized (synchronizer.getMonitor())
{
MyExecutorService exec = null;
// TODO: We should wait for them to complete their work ...
// if it's not an emergency shutdown
logger.info("Stopping workers");
for (WorkerLauncher jg : jobGenerators)
jg.stop();
for (WorkerLauncher pn : processingNodes)
pn.stop();
if (gatherer != null)
gatherer.stop();
for (WorkerLauncher s : storers)
s.stop();
if (executor != null)
{
exec = executor;
executor = null;
}
return exec;
}
}
private void nexusStopped() throws Throwable
{
state = NexusState.SHUTDOWN;
logger.info("Nexus stopped");
if (emergencyCause != null)
throw emergencyCause;
}
}
ProcessingNode.java 0000664 0000000 0000000 00000002074 13352705604 0050074 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
public abstract class ProcessingNode
extends
WorkerBase
{
public ProcessingNode(String workerName, AbortHandler abortHandler)
{
super(workerName, abortHandler);
}
public ProcessingNode(String workerName)
{
super(workerName);
}
}
ProcessingNodeFactory.java 0000664 0000000 0000000 00000002026 13352705604 0051421 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
import java.util.concurrent.BlockingQueue;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
public interface ProcessingNodeFactory
{
WorkerBase create(
AbortHandler abortHandler,
BlockingQueue inTray,
BlockingQueue processedJobs);
}
Processor.java 0000664 0000000 0000000 00000001642 13352705604 0047131 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
public interface Processor
{
/**
* Process a job and return the result.
*
* @param job
* The job to process.
* @return The result of the processed job.
*/
public Object process(Job job);
}
StorerFactory.java 0000664 0000000 0000000 00000002002 13352705604 0047747 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
import java.util.concurrent.BlockingQueue;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
public interface StorerFactory
{
WorkerBase create(
AbortHandler abortHandler,
JobTraceSet jobTraces,
BlockingQueue outTray);
}
WorkerInstantiator.java 0000664 0000000 0000000 00000001513 13352705604 0051020 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
public interface WorkerInstantiator
{
WorkerBase instantiate();
}
pnodes/ 0000775 0000000 0000000 00000000000 13352705604 0045574 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher LocalProcessingNode.java 0000664 0000000 0000000 00000010026 13352705604 0052333 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/pnodes /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.pnodes;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.CompletionService;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.Semaphore;
import org.sweble.wikitext.articlecruncher.Job;
import org.sweble.wikitext.articlecruncher.ProcessingNode;
import org.sweble.wikitext.articlecruncher.WorkerInstantiator;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.ExecutorType;
import org.sweble.wikitext.articlecruncher.utils.MyExecutorService;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
import org.sweble.wikitext.articlecruncher.utils.WorkerLauncher;
import org.sweble.wikitext.articlecruncher.utils.WorkerSynchronizer;
public class LocalProcessingNode
extends
ProcessingNode
{
private final BlockingQueue inTray;
private final BlockingQueue processedJobs;
private final int numWorkers;
private final LpnJobProcessorFactory jobProcessorFactory;
private MyExecutorService executor;
private Semaphore backPressure;
private WorkerLauncher distributor;
private WorkerLauncher gatherer;
// =========================================================================
public LocalProcessingNode(
AbortHandler abortHandler,
BlockingQueue inTray,
BlockingQueue processedJobs,
LpnJobProcessorFactory jobProcessorFactory,
int numWorkers)
{
super(getClassName(), abortHandler);
Thread.currentThread().setName(getClassName());
this.inTray = inTray;
this.processedJobs = processedJobs;
this.jobProcessorFactory = jobProcessorFactory;
this.numWorkers = numWorkers;
}
private static String getClassName()
{
return LocalProcessingNode.class.getSimpleName();
}
// =========================================================================
@Override
protected void work() throws Throwable
{
try
{
executor = new MyExecutorService(
ExecutorType.CACHED_THREAD_POOL,
getLogger());
final AbortHandler abortHandler = new AbortHandler()
{
@Override
public void notify(Throwable t)
{
stop();
}
};
WorkerSynchronizer synchronizer = new WorkerSynchronizer();
backPressure = new Semaphore(numWorkers);
final BlockingQueue> ecsQueue =
new LinkedBlockingQueue>();
distributor = new WorkerLauncher(new WorkerInstantiator()
{
@Override
public WorkerBase instantiate()
{
LpnDistributor d = new LpnDistributor(
abortHandler,
inTray,
executor.getThreadGroup(),
numWorkers,
jobProcessorFactory,
backPressure);
ecsQueue.add(d.getEcs());
return d;
}
}, abortHandler);
distributor.start(executor, synchronizer);
final CompletionService ecs = ecsQueue.take();
gatherer = new WorkerLauncher(new WorkerInstantiator()
{
@Override
public WorkerBase instantiate()
{
return new LpnGatherer(
abortHandler,
ecs,
processedJobs,
backPressure);
}
}, abortHandler);
gatherer.start(executor, synchronizer);
synchronizer.waitForAny();
}
finally
{
info("Sending kill signal to workers");
if (distributor != null)
distributor.stop();
if (gatherer != null)
gatherer.stop();
if (executor != null)
executor.shutdownAndAwaitTermination();
}
}
}
LpnDistributor.java 0000664 0000000 0000000 00000007553 13352705604 0051435 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/pnodes /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.pnodes;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Callable;
import java.util.concurrent.CompletionService;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.Semaphore;
import java.util.concurrent.SynchronousQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import org.sweble.wikitext.articlecruncher.Job;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.MyExecutorCompletionService;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
import de.fau.cs.osr.utils.WrappedException;
public class LpnDistributor
extends
WorkerBase
{
private final BlockingQueue inTray;
private final MyExecutorCompletionService execComplServ;
private final Semaphore backPressure;
private final LpnJobProcessorFactory jobProcessorFactory;
private int count = 0;
// =========================================================================
public LpnDistributor(
AbortHandler abortHandler,
BlockingQueue inTray,
ThreadGroup fatherThreadGroup,
int numWorkers,
LpnJobProcessorFactory jobProcessorFactory,
Semaphore backPressure)
{
super(getClassName(), abortHandler);
Thread.currentThread().setName(getClassName());
this.inTray = inTray;
this.backPressure = backPressure;
this.jobProcessorFactory = jobProcessorFactory;
int corePoolSize = numWorkers;
int maximumPoolSize = numWorkers;
info(getClass().getSimpleName() + " starts with a pool of " + numWorkers + " workers");
execComplServ = new MyExecutorCompletionService(
getLogger(),
corePoolSize,
maximumPoolSize,
60,
TimeUnit.SECONDS,
new SynchronousQueue(),
new RejectedExecutionHandlerImpl());
execComplServ.setThreadNameTemplate(jobProcessorFactory.getProcessorNameTemplate());
}
protected static String getClassName()
{
return LpnDistributor.class.getSimpleName();
}
// =========================================================================
@Override
protected void work() throws Throwable
{
while (true)
{
// Prevent a accumulation of processed jobs in the gatherer.
backPressure.acquire();
final Job job = inTray.take();
++count;
job.signOff(getClass(), null);
Callable worker = new LpnWorker(jobProcessorFactory, job);
execComplServ.submit(worker);
}
}
@Override
protected void after()
{
if (execComplServ != null)
execComplServ.shutdownAndAwaitTermination();
info(getClass().getSimpleName() + " counts " + count + " items");
}
// =========================================================================
public CompletionService getEcs()
{
return execComplServ;
}
// =========================================================================
private final class RejectedExecutionHandlerImpl
implements
RejectedExecutionHandler
{
@Override
public void rejectedExecution(
Runnable r,
ThreadPoolExecutor executor)
{
if (executor.isShutdown())
return;
try
{
executor.getQueue().put(r);
}
catch (InterruptedException e)
{
throw new WrappedException(e);
}
}
}
}
LpnGatherer.java 0000664 0000000 0000000 00000005602 13352705604 0050655 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/pnodes /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.pnodes;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.CompletionService;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
import java.util.concurrent.Semaphore;
import org.sweble.wikitext.articlecruncher.Job;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
public class LpnGatherer
extends
WorkerBase
{
private final CompletionService execCompServ;
private final BlockingQueue processedJobs;
private final Semaphore backPressure;
private int count = 0;
private int failureCount = 0;
private int successCount = 0;
// =========================================================================
public LpnGatherer(
AbortHandler abortHandler,
CompletionService execCompServ,
BlockingQueue processedJobs,
Semaphore backPressure)
{
super(getClassName(), abortHandler);
Thread.currentThread().setName(getClassName());
this.execCompServ = execCompServ;
this.processedJobs = processedJobs;
this.backPressure = backPressure;
}
private static String getClassName()
{
return LpnGatherer.class.getSimpleName();
}
// =========================================================================
@Override
protected void work() throws Throwable
{
while (true)
{
Future f = execCompServ.take();
++count;
try
{
Job processedJob = f.get();
switch (processedJob.getState())
{
case FAILED:
++failureCount;
break;
case HAS_RESULT:
++successCount;
break;
default:
throw new AssertionError();
}
processedJobs.put(processedJob);
backPressure.release();
}
catch (ExecutionException e)
{
error(LpnWorker.class.getSimpleName() + " failed with unhandled expection", e.getCause());
abort(e.getCause());
}
}
}
@Override
protected void after()
{
info(getClass().getSimpleName() + " counts " + count + " items");
info(getClass().getSimpleName() + " counts " + failureCount + " items for which processing failed");
info(getClass().getSimpleName() + " counts " + successCount + " items which were successfully processed");
}
}
LpnJobProcessorFactory.java 0000664 0000000 0000000 00000001605 13352705604 0053055 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/pnodes /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.pnodes;
import org.sweble.wikitext.articlecruncher.Processor;
public interface LpnJobProcessorFactory
{
public Processor createProcessor();
public String getProcessorNameTemplate();
}
LpnWorker.java 0000664 0000000 0000000 00000003364 13352705604 0050370 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/pnodes /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.pnodes;
import java.util.concurrent.Callable;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.sweble.wikitext.articlecruncher.Job;
import org.sweble.wikitext.articlecruncher.Processor;
final class LpnWorker
implements
Callable
{
private static final Logger logger = LoggerFactory.getLogger(LpnWorker.class.getSimpleName());
private final LpnJobProcessorFactory jobProcessorFactory;
private final Job job;
// =========================================================================
LpnWorker(LpnJobProcessorFactory jobProcessorFactory, Job jobHistory)
{
this.jobProcessorFactory = jobProcessorFactory;
this.job = jobHistory;
}
// =========================================================================
@Override
public Job call()
{
try
{
job.signOff(getClass(), null);
Processor processor = jobProcessorFactory.createProcessor();
job.processed(processor.process(job));
}
catch (Exception t)
{
logger.warn("Processing failed with exception", t);
job.failed(t);
}
return job;
}
}
storers/ 0000775 0000000 0000000 00000000000 13352705604 0046005 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher DummyStorer.java 0000664 0000000 0000000 00000003714 13352705604 0051147 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/storers /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.storers;
import java.util.concurrent.BlockingQueue;
import org.sweble.wikitext.articlecruncher.Job;
import org.sweble.wikitext.articlecruncher.JobTrace;
import org.sweble.wikitext.articlecruncher.JobTraceSet;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
public class DummyStorer
extends
WorkerBase
{
private final JobTraceSet jobTraces;
private final BlockingQueue outTray;
// =========================================================================
public DummyStorer(
AbortHandler abortHandler,
JobTraceSet jobTraces,
BlockingQueue outTray)
{
super(DummyStorer.class.getSimpleName(), abortHandler);
this.outTray = outTray;
this.jobTraces = jobTraces;
}
// =========================================================================
int count = 0;
@Override
protected void work() throws Throwable
{
while (true)
{
Job job = outTray.take();
++count;
// Do nothing
JobTrace trace = job.getTrace();
trace.signOff(getClass(), null);
if (!jobTraces.remove(trace))
throw new AssertionError("Missing job trace");
}
}
@Override
protected void after()
{
info(getClass().getSimpleName() + " counts " + count + " items");
}
}
utils/ 0000775 0000000 0000000 00000000000 13352705604 0045444 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher AbortHandler.java 0000664 0000000 0000000 00000001415 13352705604 0050655 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/utils /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.utils;
public interface AbortHandler
{
void notify(Throwable t);
}
DaemonThreadFactory.java 0000664 0000000 0000000 00000004124 13352705604 0052173 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/utils /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.utils;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.atomic.AtomicInteger;
public class DaemonThreadFactory
implements
ThreadFactory
{
private final AtomicInteger threadNumber = new AtomicInteger(1);
private final ThreadGroup group;
private String threadNameTemplate;
// =========================================================================
public DaemonThreadFactory(String poolName)
{
this(poolName, Thread.currentThread().getThreadGroup());
}
public DaemonThreadFactory(String poolName, ThreadGroup group)
{
if (group == null)
group = new ThreadGroup(poolName);
this.group = group;
this.threadNameTemplate = String.format("%s-%%02d", poolName);
}
// =========================================================================
public void setThreadNameTemplate(String threadNameTemplate)
{
this.threadNameTemplate = threadNameTemplate;
}
// =========================================================================
@Override
public Thread newThread(Runnable runnable)
{
String name = String.format(threadNameTemplate, threadNumber.getAndIncrement());
Thread thread = new Thread(group, runnable, name);
if (!thread.isDaemon())
thread.setDaemon(true);
if (thread.getPriority() != Thread.NORM_PRIORITY)
thread.setPriority(Thread.NORM_PRIORITY);
return thread;
}
public ThreadGroup getThreadGroup()
{
return group;
}
}
ExecutorType.java 0000664 0000000 0000000 00000001401 13352705604 0050743 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/utils /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.utils;
public enum ExecutorType
{
CACHED_THREAD_POOL
}
MyExecutorCompletionService.java 0000664 0000000 0000000 00000010101 13352705604 0053757 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/utils /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.utils;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Callable;
import java.util.concurrent.CompletionService;
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.Future;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.TimeUnit;
import org.slf4j.Logger;
public class MyExecutorCompletionService
implements
CompletionService
{
private final ExecutorCompletionService execComplServ;
private final MyExecutorService executor;
// =========================================================================
public MyExecutorCompletionService(ExecutorType type, Logger logger)
{
this(type, logger, null);
}
public MyExecutorCompletionService(
ExecutorType type,
Logger logger,
ThreadGroup threadGroup)
{
executor = new MyExecutorService(type, logger, threadGroup);
execComplServ = new ExecutorCompletionService(executor);
}
public MyExecutorCompletionService(
Logger logger,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue)
{
this(logger,
corePoolSize,
maximumPoolSize,
keepAliveTime,
unit,
workQueue,
(ThreadGroup) null);
}
public MyExecutorCompletionService(
Logger logger,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
RejectedExecutionHandler handler)
{
this(logger,
corePoolSize,
maximumPoolSize,
keepAliveTime,
unit,
workQueue,
null,
handler);
}
public MyExecutorCompletionService(
Logger logger,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
ThreadGroup threadGroup)
{
executor =
new MyExecutorService(
logger,
maximumPoolSize,
maximumPoolSize,
keepAliveTime,
unit,
workQueue,
threadGroup);
execComplServ = new ExecutorCompletionService(executor);
}
public MyExecutorCompletionService(
Logger logger,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
ThreadGroup threadGroup,
RejectedExecutionHandler handler)
{
executor = new MyExecutorService(
logger,
maximumPoolSize,
maximumPoolSize,
keepAliveTime,
unit,
workQueue,
threadGroup,
handler);
execComplServ = new ExecutorCompletionService(executor);
}
// =========================================================================
public MyExecutorService getExecutor()
{
return executor;
}
public void shutdownAndAwaitTermination()
{
executor.shutdownAndAwaitTermination();
}
public void setThreadNameTemplate(String threadNameTemplate)
{
executor.setThreadNameTemplate(threadNameTemplate);
}
// =========================================================================
public Future submit(Callable task)
{
return execComplServ.submit(task);
}
public Future submit(Runnable task, V result)
{
return execComplServ.submit(task, result);
}
public Future take() throws InterruptedException
{
return execComplServ.take();
}
public Future poll()
{
return execComplServ.poll();
}
public Future poll(long timeout, TimeUnit unit) throws InterruptedException
{
return execComplServ.poll(timeout, unit);
}
}
MyExecutorService.java 0000664 0000000 0000000 00000014731 13352705604 0051742 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/utils /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.utils;
import java.util.Collection;
import java.util.List;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import org.slf4j.Logger;
public class MyExecutorService
implements
ExecutorService
{
private static final long SHUTDOWN_TIMEOUT_IN_SECONDS = 60 * 5;
// =========================================================================
private final Logger logger;
private final ExecutorService executor;
private final DaemonThreadFactory threadFactory;
// =========================================================================
public MyExecutorService(ExecutorType type, Logger logger)
{
this(type, logger, null);
}
public MyExecutorService(
ExecutorType type,
Logger logger,
ThreadGroup threadGroup)
{
this.logger = logger;
this.threadFactory = new DaemonThreadFactory(
logger.getName(), threadGroup);
switch (type)
{
case CACHED_THREAD_POOL:
this.executor = Executors.newCachedThreadPool(threadFactory);
break;
default:
throw new IllegalArgumentException("Invalid executor type");
}
}
public MyExecutorService(
Logger logger,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue)
{
this(logger, corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, null);
}
public MyExecutorService(
Logger logger,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
ThreadGroup threadGroup)
{
this.logger = logger;
this.threadFactory = new DaemonThreadFactory(
logger.getName(), threadGroup);
this.executor = new ThreadPoolExecutor(
corePoolSize,
maximumPoolSize,
60,
TimeUnit.SECONDS,
workQueue,
threadFactory);
}
public MyExecutorService(
Logger logger,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
ThreadGroup threadGroup,
RejectedExecutionHandler handler)
{
this.logger = logger;
this.threadFactory = new DaemonThreadFactory(
logger.getName(), threadGroup);
this.executor = new ThreadPoolExecutor(
corePoolSize,
maximumPoolSize,
60,
TimeUnit.SECONDS,
workQueue,
threadFactory,
handler);
}
// =========================================================================
public ExecutorService getExecutor()
{
return executor;
}
public ThreadGroup getThreadGroup()
{
return threadFactory.getThreadGroup();
}
public void setThreadNameTemplate(String threadNameTemplate)
{
this.threadFactory.setThreadNameTemplate(threadNameTemplate);
}
// =========================================================================
/*
* Taken from usage example from:
* http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html
*/
public synchronized void shutdownAndAwaitTermination()
{
logger.info("Shutting down workers");
// Disable new tasks from being submitted
executor.shutdown();
try
{
logger.info("Waiting for workers to terminate");
if (!executor.awaitTermination(SHUTDOWN_TIMEOUT_IN_SECONDS, TimeUnit.SECONDS))
{
logger.error("Workers are not responding to shutdown! Forcing shutdown.");
// Cancel currently executing tasks
executor.shutdownNow();
// Wait a while for tasks to respond to being cancelled
if (!executor.awaitTermination(SHUTDOWN_TIMEOUT_IN_SECONDS, TimeUnit.SECONDS))
logger.error("Workers did not terminate!");
}
}
catch (InterruptedException ie)
{
logger.error("Worker shutdown interrupted! Forcing shutdown.");
// (Re-)Cancel if current thread also interrupted
executor.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
}
// =========================================================================
@Override
public void execute(Runnable command)
{
executor.execute(command);
}
@Override
public void shutdown()
{
executor.shutdown();
}
@Override
public List shutdownNow()
{
return executor.shutdownNow();
}
@Override
public boolean isShutdown()
{
return executor.isShutdown();
}
@Override
public boolean isTerminated()
{
return executor.isTerminated();
}
@Override
public boolean awaitTermination(long timeout, TimeUnit unit) throws InterruptedException
{
return executor.awaitTermination(timeout, unit);
}
@Override
public Future submit(Callable task)
{
return executor.submit(task);
}
@Override
public Future submit(Runnable task, T result)
{
return executor.submit(task, result);
}
@Override
public Future submit(Runnable task)
{
return executor.submit(task);
}
@Override
public List> invokeAll(Collection> tasks) throws InterruptedException
{
return executor.invokeAll(tasks);
}
@Override
public List> invokeAll(
Collection> tasks,
long timeout,
TimeUnit unit) throws InterruptedException
{
return executor.invokeAll(tasks, timeout, unit);
}
@Override
public T invokeAny(Collection> tasks) throws InterruptedException, ExecutionException
{
return executor.invokeAny(tasks);
}
@Override
public T invokeAny(
Collection> tasks,
long timeout,
TimeUnit unit) throws InterruptedException, ExecutionException, TimeoutException
{
return executor.invokeAny(tasks, timeout, unit);
}
}
SpeedMeter.java 0000664 0000000 0000000 00000006257 13352705604 0050356 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/utils /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.utils;
import de.fau.cs.osr.utils.RingBuffer;
public class SpeedMeter
{
private final RingBuffer speedBuffer;
private final float progressUpdateInterval;
private final float speedUpdateInterval;
private long lastProgressUpdate = System.currentTimeMillis();
private long lastSpeedUpdate = lastProgressUpdate;
private long lastBytesRead = 0;
private float currentProgress;
private float currentSpeed = Float.NaN;
private float avgSpeed = Float.NaN;
private float eta = Float.POSITIVE_INFINITY;
// =========================================================================
public SpeedMeter(
float progressUpdateIntervalInSeconds,
float speedUpdateIntervalInSeconds,
float etaWindowLengthInSeconds)
{
this.progressUpdateInterval = progressUpdateIntervalInSeconds;
this.speedUpdateInterval = speedUpdateIntervalInSeconds;
int capacity = (int) (etaWindowLengthInSeconds / speedUpdateInterval);
if (capacity <= 0 || capacity > 4096)
throw new IllegalArgumentException("Cannot realize ETA configuration");
this.speedBuffer = new RingBuffer(
capacity);
}
// =========================================================================
public void update(long bytesRead, long fileLength)
{
long thisTime = System.currentTimeMillis();
updateSpeed(thisTime, bytesRead, fileLength);
updateProgress(thisTime, bytesRead, fileLength);
}
public float getCurrentProgress()
{
return currentProgress;
}
public float getCurrentSpeed()
{
return currentSpeed;
}
public float getAvgSpeed()
{
return avgSpeed;
}
public float getEta()
{
return eta;
}
// =========================================================================
private void updateProgress(long now, long bytesRead, long fileLength)
{
float elapsed = (now - lastProgressUpdate) / 1000.f;
if (elapsed > progressUpdateInterval)
{
lastProgressUpdate = now;
currentProgress = (float) bytesRead / fileLength * 100.0f;
}
}
private void updateSpeed(long now, long bytesRead, long fileLength)
{
float elapsed = (now - lastSpeedUpdate) / 1000.f;
if (elapsed > speedUpdateInterval)
{
lastSpeedUpdate = now;
long processed = bytesRead - lastBytesRead;
lastBytesRead = bytesRead;
currentSpeed = processed / elapsed;
speedBuffer.add(currentSpeed);
avgSpeed = 0;
for (float speed : speedBuffer)
avgSpeed += speed;
avgSpeed /= speedBuffer.size();
if (speedBuffer.size() > 10)
eta = (fileLength - bytesRead) / avgSpeed / 60.0f;
}
}
}
WorkerBase.java 0000664 0000000 0000000 00000005256 13352705604 0050363 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/utils /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.utils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public abstract class WorkerBase
{
private final String workerName;
private final AbortHandler abortHandler;
private final Logger logger;
private WorkerLauncher launcher;
// =========================================================================
public WorkerBase(String workerName)
{
this(workerName, new AbortHandler()
{
@Override
public void notify(Throwable e)
{
// Does nothing
}
});
}
public WorkerBase(String workerName, AbortHandler abortHandler)
{
this.workerName = workerName;
this.abortHandler = abortHandler;
this.logger = LoggerFactory.getLogger(workerName);
}
// =========================================================================
protected abstract void work() throws Throwable;
protected void after()
{
}
protected void abort(Throwable e)
{
abortHandler.notify(e);
}
protected void stop()
{
launcher.stop();
}
// =========================================================================
protected void setLauncher(WorkerLauncher launcher)
{
this.launcher = launcher;
}
protected String getWorkerName()
{
return workerName;
}
protected Logger getLogger()
{
return logger;
}
protected void trace(String message)
{
logger.trace(message);
}
protected void trace(String message, Throwable t)
{
logger.trace(message, t);
}
protected void debug(String message)
{
logger.debug(message);
}
protected void debug(String message, Throwable t)
{
logger.debug(message, t);
}
protected void info(String message)
{
logger.info(message);
}
protected void info(String message, Throwable t)
{
logger.info(message, t);
}
protected void warn(String message)
{
logger.warn(message);
}
protected void warn(String message, Throwable t)
{
logger.warn(message, t);
}
protected void error(String message)
{
logger.error(message);
}
protected void error(String message, Throwable t)
{
logger.error(message, t);
}
}
WorkerLauncher.java 0000664 0000000 0000000 00000012536 13352705604 0051251 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/utils /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.utils;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.sweble.wikitext.articlecruncher.WorkerInstantiator;
import de.fau.cs.osr.utils.WrappedException;
public class WorkerLauncher
{
public enum WorkerState
{
INITIALIZED,
RUNNING,
POISONED,
STOPPED,
}
private static final Logger logger = LoggerFactory.getLogger(WorkerLauncher.class.getName());
private final Object kickOffLock = new Object();
private final WorkerInstantiator workerInstantiator;
private final AbortHandler abortHandler;
private String workerName;
private Future future;
private WorkerState state;
private WorkerSynchronizer synchronizer;
// =========================================================================
public WorkerLauncher(
WorkerInstantiator workerInstantiator,
AbortHandler abortHandler)
{
this.workerInstantiator = workerInstantiator;
this.abortHandler = abortHandler;
//this.logger = Logger.getLogger(workerName);
this.state = WorkerState.INITIALIZED;
}
// =========================================================================
public final synchronized void start(ExecutorService executor)
{
start(executor, null);
}
public final synchronized void start(
ExecutorService executor,
WorkerSynchronizer synchronizer)
{
synchronized (state)
{
if (state != WorkerState.INITIALIZED)
{
throw new IllegalStateException("start() can be called only once");
}
else
{
// Make sure that this.future is set before the worker can kick-off
synchronized (kickOffLock)
{
this.state = WorkerState.RUNNING;
this.synchronizer = synchronizer;
this.future = executor.submit(new WorkerRunnable());
}
}
}
}
public final synchronized void stop()
{
synchronized (state)
{
switch (state)
{
case RUNNING:
logger.info("Sending stop signal to worker " + workerName);
state = WorkerState.POISONED;
future.cancel(true);
break;
/*
case POISONED:
logger.warn("Already sent stop signal to worker " + workerName);
break;
case STOPPED:
logger.warn("Worker " + workerName + " already terminated");
break;
case INITIALIZED:
throw new IllegalStateException("stop() can only be called after start()");
*/
default:
break;
}
}
}
public final synchronized void await() throws InterruptedException, ExecutionException
{
await(Long.MAX_VALUE, null);
}
public final synchronized boolean await(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException
{
Future f;
synchronized (state)
{
if (state == WorkerState.INITIALIZED)
throw new IllegalStateException("await() can only be called after start()");
f = future;
}
if (f != null)
{
try
{
if (timeout == Long.MAX_VALUE && unit == null)
{
future.get();
}
else
{
future.get(timeout, unit);
}
}
catch (TimeoutException e)
{
// timed out
return false;
}
}
else
{
logger.warn("Worker " + workerName + " already terminated");
}
// joined with future
return true;
}
// =========================================================================
private final class WorkerRunnable
implements
Runnable
{
@Override
public void run()
{
synchronized (kickOffLock)
{
}
WorkerBase worker = workerInstantiator.instantiate();
workerName = worker.getWorkerName();
worker.setLauncher(WorkerLauncher.this);
try
{
if (synchronizer != null)
synchronizer.oneStarted();
logger.info(workerName + " starting");
try
{
worker.work();
}
catch (WrappedException e)
{
throw e.getCause();
}
}
catch (InterruptedException e)
{
boolean unexpected = false;
synchronized (state)
{
// Don't call abort inside lock!
if (state != WorkerState.POISONED)
unexpected = true;
}
if (unexpected)
{
logger.error(workerName + " interrupted unexpectedly", e);
abortHandler.notify(e);
}
}
catch (Throwable t)
{
logger.error(workerName + " terminated by exception", t);
abortHandler.notify(t);
}
finally
{
try
{
worker.after();
}
catch (Throwable t)
{
logger.error(workerName + ".after() threw exception", t);
}
synchronized (state)
{
state = WorkerState.STOPPED;
}
logger.info(workerName + " stopped");
if (synchronizer != null)
synchronizer.oneStopped();
}
}
}
}
WorkerSynchronizer.java 0000664 0000000 0000000 00000004417 13352705604 0052204 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/main/java/org/sweble/wikitext/articlecruncher/utils /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.articlecruncher.utils;
public class WorkerSynchronizer
{
private final Object lock = new Object();
private final Object goLock = new Object();
private int running = 0;
private boolean oneStopped = false;
private boolean abort = false;
private boolean isSync = false;
private boolean go = false;
// =========================================================================
public WorkerSynchronizer()
{
}
// =========================================================================
public void oneStarted() throws InterruptedException
{
synchronized (lock)
{
++running;
lock.notify();
}
synchronized (goLock)
{
while (!go)
goLock.wait();
}
}
public void oneStopped()
{
synchronized (lock)
{
--running;
oneStopped = true;
lock.notify();
}
}
public void abort()
{
synchronized (lock)
{
abort = true;
lock.notify();
}
}
public void waitForAll(int numWaitingFor) throws InterruptedException
{
synchronized (lock)
{
while (running < numWaitingFor)
lock.wait();
}
synchronized (goLock)
{
go = true;
goLock.notifyAll();
}
synchronized (lock)
{
while (running > 0 && !abort)
lock.wait();
isSync = true;
}
}
public void waitForAny() throws InterruptedException
{
synchronized (goLock)
{
go = true;
goLock.notifyAll();
}
synchronized (lock)
{
while (!oneStopped && !abort)
lock.wait();
isSync = true;
}
}
public boolean isSynchronized()
{
return isSync;
}
public boolean isAborted()
{
return abort;
}
public Object getMonitor()
{
return lock;
}
}
test/ 0000775 0000000 0000000 00000000000 13352705604 0034321 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src java/ 0000775 0000000 0000000 00000000000 13352705604 0035242 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/test CruncherTest.java 0000664 0000000 0000000 00000007740 13352705604 0040526 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/test/java /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
import java.util.Set;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.atomic.AtomicLong;
import org.junit.Before;
import org.junit.Test;
import org.sweble.wikitext.articlecruncher.Job;
import org.sweble.wikitext.articlecruncher.JobGeneratorFactory;
import org.sweble.wikitext.articlecruncher.JobTrace;
import org.sweble.wikitext.articlecruncher.Nexus;
import org.sweble.wikitext.articlecruncher.ProcessingNodeFactory;
import org.sweble.wikitext.articlecruncher.StorerFactory;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
public class CruncherTest
extends
CruncherTestBase
{
private Nexus nexus;
private AtomicLong processed = new AtomicLong(0);
private long failAfter = -1;
// =========================================================================
@Before
public void before() throws Throwable
{
nexus = new Nexus();
nexus.setUp(
16, /* in tray capacity */
16, /* processed jobs capacity */
16 /* out tray capacity */);
JobGeneratorFactory jobFactory = createJobFactory();
nexus.addJobGenerator(jobFactory);
ProcessingNodeFactory pnFactory = createPnFactory();
nexus.addProcessingNode(pnFactory);
StorerFactory storerFactory = createStorerFactory();
nexus.addStorer(storerFactory);
}
// =========================================================================
@Test
public void testWithoutFailing() throws Throwable
{
nexus.start();
assertEquals(NUM_JOBS_TO_GENERATE, generated.get());
assertEquals(NUM_JOBS_TO_GENERATE, processed.get());
assertEquals(NUM_JOBS_TO_GENERATE, stored.get());
Set jobTraces = nexus.getJobTraces();
assertTrue(jobTraces.isEmpty());
}
@Test
public void testWithFailing() throws Throwable
{
failAfter = (NUM_JOBS_TO_GENERATE * 3) / 4;
try
{
nexus.start();
}
catch (RuntimeException e)
{
assertEquals("Die!", e.getMessage());
}
Set jobTraces = nexus.getJobTraces();
assertEquals(generated.get(), processed.get() + jobTraces.size());
assertEquals(generated.get(), stored.get() + jobTraces.size());
/*
System.out.println("numJobsToGenerate: " + NUM_JOBS_TO_GENERATE);
System.out.println("generated: " + generated.get());
System.out.println("processed: " + processed.get());
System.out.println("stored: " + stored.get());
System.out.println("job traces left: " + jobTraces.size());
*/
}
// =========================================================================
private ProcessingNodeFactory createPnFactory()
{
return new ProcessingNodeFactory()
{
@Override
public WorkerBase create(
final AbortHandler abortHandler,
final BlockingQueue inTray,
final BlockingQueue processedJobs)
{
return new WorkerBase("ProcessingNode", abortHandler)
{
@Override
protected void work() throws InterruptedException
{
while (true)
{
Job job = inTray.take();
job.signOff(getClass(), null);
// do process.
if (failAfter >= 0 && generated.get() > failAfter)
throw new RuntimeException("Die!");
processed.incrementAndGet();
job.processed((Object) null);
processedJobs.put(job);
}
}
};
}
};
}
}
CruncherTestBase.java 0000664 0000000 0000000 00000006176 13352705604 0041323 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/test/java /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.atomic.AtomicLong;
import org.sweble.wikitext.articlecruncher.Job;
import org.sweble.wikitext.articlecruncher.JobGeneratorFactory;
import org.sweble.wikitext.articlecruncher.JobTrace;
import org.sweble.wikitext.articlecruncher.JobTraceSet;
import org.sweble.wikitext.articlecruncher.StorerFactory;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
import de.fau.cs.osr.utils.WrappedException;
public class CruncherTestBase
{
protected static final long NUM_JOBS_TO_GENERATE = (long) Math.pow(2, 20);
protected AtomicLong generated = new AtomicLong(0);
protected AtomicLong stored = new AtomicLong(0);
// =========================================================================
protected static final class TestJob
extends
Job
{
}
// =========================================================================
protected JobGeneratorFactory createJobFactory()
{
return new JobGeneratorFactory()
{
@Override
public WorkerBase create(
final AbortHandler abortHandler,
final BlockingQueue inTray,
final JobTraceSet jobTraces)
{
try
{
return new WorkerBase("JobGenerator", abortHandler)
{
@Override
protected void work() throws InterruptedException
{
for (long i = 0; i < NUM_JOBS_TO_GENERATE; ++i)
{
// generate jobs out of thin air.
Job job = new TestJob();
generated.incrementAndGet();
JobTrace trace = job.getTrace();
trace.signOff(getClass(), null);
jobTraces.add(trace);
inTray.put(job);
}
}
};
}
catch (Exception e)
{
throw new WrappedException(e);
}
}
};
}
protected StorerFactory createStorerFactory()
{
return new StorerFactory()
{
@Override
public WorkerBase create(
final AbortHandler abortHandler,
final JobTraceSet jobTraces,
final BlockingQueue outTray)
{
return new WorkerBase("Storer", abortHandler)
{
@Override
protected void work() throws InterruptedException
{
while (true)
{
TestJob job = (TestJob) outTray.take();
JobTrace trace = job.getTrace();
trace.signOff(getClass(), null);
// do store.
stored.incrementAndGet();
if (!jobTraces.remove(trace))
throw new AssertionError("Missing job trace");
}
}
};
}
};
}
}
LpnCruncherTest.java 0000664 0000000 0000000 00000007100 13352705604 0041166 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-article-cruncher/src/test/java /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
import java.util.Set;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.atomic.AtomicLong;
import org.junit.Before;
import org.junit.Test;
import org.sweble.wikitext.articlecruncher.Job;
import org.sweble.wikitext.articlecruncher.JobGeneratorFactory;
import org.sweble.wikitext.articlecruncher.JobTrace;
import org.sweble.wikitext.articlecruncher.Nexus;
import org.sweble.wikitext.articlecruncher.ProcessingNodeFactory;
import org.sweble.wikitext.articlecruncher.Processor;
import org.sweble.wikitext.articlecruncher.StorerFactory;
import org.sweble.wikitext.articlecruncher.pnodes.LocalProcessingNode;
import org.sweble.wikitext.articlecruncher.pnodes.LpnJobProcessorFactory;
import org.sweble.wikitext.articlecruncher.utils.AbortHandler;
import org.sweble.wikitext.articlecruncher.utils.WorkerBase;
public class LpnCruncherTest
extends
CruncherTestBase
{
private Nexus nexus;
private static final int NUM_WORKERS = 256;
private AtomicLong processed = new AtomicLong(0);
// =========================================================================
@Before
public void before() throws Throwable
{
nexus = new Nexus();
nexus.setUp(
16, /* in tray capacity */
16, /* processed jobs capacity */
16 /* out tray capacity */);
JobGeneratorFactory jobFactory = createJobFactory();
nexus.addJobGenerator(jobFactory);
ProcessingNodeFactory pnFactory = createPnFactory();
nexus.addProcessingNode(pnFactory);
StorerFactory storerFactory = createStorerFactory();
nexus.addStorer(storerFactory);
}
// =========================================================================
@Test
public void test() throws Throwable
{
nexus.start();
assertEquals(NUM_JOBS_TO_GENERATE, generated.get());
assertEquals(NUM_JOBS_TO_GENERATE, processed.get());
assertEquals(NUM_JOBS_TO_GENERATE, stored.get());
Set jobTraces = nexus.getJobTraces();
assertTrue(jobTraces.isEmpty());
}
// =========================================================================
private ProcessingNodeFactory createPnFactory()
{
return new ProcessingNodeFactory()
{
@Override
public WorkerBase create(
AbortHandler abortHandler,
BlockingQueue inTray,
BlockingQueue processedJobs)
{
return new LocalProcessingNode(
abortHandler,
inTray,
processedJobs,
createLpnFactory(),
NUM_WORKERS);
}
};
}
private LpnJobProcessorFactory createLpnFactory()
{
return new LpnJobProcessorFactory()
{
@Override
public Processor createProcessor()
{
return new Processor()
{
@Override
public Object process(Job job)
{
job.signOff(getClass(), null);
processed.incrementAndGet();
return null;
}
};
}
@Override
public String getProcessorNameTemplate()
{
return "Processor-%02d";
}
};
}
}
sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/ 0000775 0000000 0000000 00000000000 13352705604 0031530 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/pom.xml 0000664 0000000 0000000 00000021000 13352705604 0033036 0 ustar 00root root 0000000 0000000
4.0.0
org.sweble.wikitext
sweble-wikitext-components-parent
3.1.9
swc-dumpreader
jar
SWC - Sweble Dump Reader
2012
A small library for reading in Wikipedia dumps.
../
src/main/schema
${project.build.directory}/generated-sources
org.aspectj
aspectjrt
org.apache.commons
commons-compress
commons-io
commons-io
joda-time
joda-time
org.slf4j
slf4j-api
de.fau.cs.osr.utils
utils
junit
junit
test
${schema-sources.directory}
com.mycila
license-maven-plugin
src/main/schema/xml.xsd
src/main/schema/export-*.xsd
org.codehaus.mojo
build-helper-maven-plugin
add-source
generate-sources
add-source
${schema-sources.directory}
${generated-sources.directory}
org.jvnet.jaxb2.maven2
maven-jaxb2-plugin
${schema-sources.directory}
${generated-sources.directory}
${schema-sources.directory}/catalog.xml
true
true
compile-mw-schema-0.5
generate
export-0.5.xsd
org.sweble.wikitext.dumpreader.export_0_5
compile-mw-schema-0.6
generate
export-0.6-fixed.xsd
org.sweble.wikitext.dumpreader.export_0_6
compile-mw-schema-0.7
generate
export-0.7.xsd
org.sweble.wikitext.dumpreader.export_0_7
compile-mw-schema-0.8
generate
export-0.8.xsd
org.sweble.wikitext.dumpreader.export_0_8
compile-mw-schema-0.9
generate
export-0.9.xsd
org.sweble.wikitext.dumpreader.export_0_9
compile-mw-schema-0.10
generate
export-0.10.xsd
org.sweble.wikitext.dumpreader.export_0_10
org.codehaus.mojo
aspectj-maven-plugin
false
maven-eclipse-plugin
false
1.5
de.thetaphi
forbiddenapis
commons-io-unsafe-1.4
maven-shade-plugin
package
shade
true
jar-with-dependencies
sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/ 0000775 0000000 0000000 00000000000 13352705604 0032317 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/ 0000775 0000000 0000000 00000000000 13352705604 0033243 5 ustar 00root root 0000000 0000000 java/ 0000775 0000000 0000000 00000000000 13352705604 0034105 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main org/ 0000775 0000000 0000000 00000000000 13352705604 0034674 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java sweble/ 0000775 0000000 0000000 00000000000 13352705604 0036155 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org wikitext/ 0000775 0000000 0000000 00000000000 13352705604 0040025 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble dumpreader/ 0000775 0000000 0000000 00000000000 13352705604 0042155 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble/wikitext CountingInputStream.java 0000664 0000000 0000000 00000004262 13352705604 0047006 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble/wikitext/dumpreader /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.dumpreader;
import java.io.IOException;
import java.io.InputStream;
public final class CountingInputStream
extends
InputStream
{
private InputStream in;
private long count;
// =========================================================================
public CountingInputStream(InputStream inputStream)
{
this.in = inputStream;
this.count = 0;
}
// =========================================================================
public long getCount()
{
return count;
}
// =========================================================================
@Override
public int read() throws IOException
{
int read = in.read();
if (read != -1)
++count;
return read;
}
@Override
public int read(byte[] b) throws IOException
{
int read = in.read(b);
if (read != -1)
count += read;
return read;
}
@Override
public int read(byte[] b, int off, int len) throws IOException
{
int read = in.read(b, off, len);
if (read != -1)
count += read;
return read;
}
@Override
public long skip(long n) throws IOException
{
long skipped = in.skip(n);
if (skipped != -1)
++count;
return skipped;
}
@Override
public int available() throws IOException
{
return in.available();
}
@Override
public void close() throws IOException
{
in.close();
}
@Override
public void mark(int readlimit)
{
in.mark(readlimit);
}
@Override
public void reset() throws IOException
{
in.reset();
}
@Override
public boolean markSupported()
{
return in.markSupported();
}
}
DumpReader.java 0000664 0000000 0000000 00000025341 13352705604 0045055 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble/wikitext/dumpreader /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.dumpreader;
import java.io.BufferedInputStream;
import java.io.Closeable;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.net.URL;
import java.nio.charset.Charset;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Unmarshaller;
import javax.xml.bind.ValidationEvent;
import javax.xml.bind.ValidationEventHandler;
import javax.xml.bind.ValidationEventLocator;
import javax.xml.stream.FactoryConfigurationError;
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamReader;
import javax.xml.validation.Schema;
import javax.xml.validation.SchemaFactory;
import org.apache.commons.compress.compressors.bzip2.BZip2CompressorInputStream;
import org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream;
import org.apache.commons.io.IOUtils;
import org.slf4j.Logger;
import org.xml.sax.SAXException;
import de.fau.cs.osr.utils.WrappedException;
public abstract class DumpReader
implements
Closeable
{
private final InputStream dumpInputStream;
private final String dumpUri;
private final Logger logger;
private final XMLStreamReader xmlStreamReader;
private final Unmarshaller unmarshaller;
private final ExportSchemaVersion schemaVersion;
private CountingInputStream decompressedInputStream;
private CountingInputStream compressedInputStream;
private long fileLength;
private long parsedCount;
private boolean decompress;
private static final int LOOKAHEAD = 4096;
// =========================================================================
/**
* Use a constructor that expects an encoding to prevent bug in xerces
* parser code.
*
* @deprecated
*/
public DumpReader(File dumpFile, Logger logger) throws JAXBException, FactoryConfigurationError, XMLStreamException, IOException, SAXException
{
this(new FileInputStream(dumpFile), dumpFile.getAbsolutePath(), logger);
fileLength = dumpFile.length();
}
/**
* Use a constructor that expects an encoding to prevent bug in xerces
* parser code.
*
* @deprecated
*/
public DumpReader(InputStream is, String url, Logger logger) throws JAXBException, FactoryConfigurationError, XMLStreamException, IOException, SAXException
{
this(is, null, url, logger, true);
}
/**
* Use a constructor that expects an encoding to prevent bug in xerces
* parser code.
*
* @deprecated
*/
public DumpReader(
InputStream is,
String url,
Logger logger,
boolean useSchema) throws JAXBException, FactoryConfigurationError, XMLStreamException, IOException, SAXException
{
this(is, null, url, logger, useSchema);
}
public DumpReader(
InputStream is,
Charset encoding,
String url,
Logger logger,
boolean useSchema) throws JAXBException, FactoryConfigurationError, XMLStreamException, IOException, SAXException
{
this.dumpInputStream = is;
this.dumpUri = url;
this.logger = logger;
logger.info("Setting up parser for file " + dumpUri);
getDumpInputStream();
schemaVersion = determineExportVersion();
unmarshaller = createUnmarshaller(schemaVersion.getContextPath());
installCallbacks();
if (useSchema)
setSchema(DumpReader.class.getResource(schemaVersion.getSchema()));
xmlStreamReader = getXmlStreamReader(encoding);
fileLength = -1;
parsedCount = 0;
}
// =========================================================================
public void unmarshal() throws JAXBException, XMLStreamException
{
try
{
unmarshaller.unmarshal(xmlStreamReader);
}
finally
{
closeStreams();
}
}
@Override
public void close() throws IOException
{
closeStreams();
}
private void closeStreams()
{
IOUtils.closeQuietly(decompressedInputStream);
IOUtils.closeQuietly(compressedInputStream);
IOUtils.closeQuietly(dumpInputStream);
}
public long getFileSize()
{
return fileLength;
}
public long getDecompressedBytesRead() throws IOException
{
return decompressedInputStream.getCount();
}
public long getCompressedBytesRead() throws IOException
{
if (decompress)
{
return compressedInputStream.getCount();
}
else
{
return getDecompressedBytesRead();
}
}
public long getParsedCount()
{
return parsedCount;
}
// =========================================================================
protected abstract void processPage(Object mediaWiki, Object page);
protected boolean processRevision(Object page, Object revision)
{
// Add by default
return true;
}
protected boolean processEvent(
ValidationEvent ve,
ValidationEventLocator vel)
{
logger.warn(String.format(
"%s:%d:%d: %s",
dumpUri,
vel.getLineNumber(),
vel.getColumnNumber(),
ve.getMessage()));
return true;
}
// =========================================================================
private void handlePage(Object mediaWiki, Object page)
{
++parsedCount;
processPage(mediaWiki, page);
}
private boolean handleRevision(Object page, Object revision)
{
return processRevision(page, revision);
}
protected boolean handleEvent(ValidationEvent ve, ValidationEventLocator vel)
{
return processEvent(ve, vel);
}
// =========================================================================
private void getDumpInputStream() throws IOException
{
InputStream decomp;
if (dumpUri.endsWith(".bz2"))
{
decompress = true;
compressedInputStream = new CountingInputStream(dumpInputStream);
decomp = new BZip2CompressorInputStream(compressedInputStream, true);
}
else if (dumpUri.endsWith(".gz"))
{
decompress = true;
compressedInputStream = new CountingInputStream(dumpInputStream);
decomp = new GzipCompressorInputStream(compressedInputStream);
}
else
{
decompress = false;
decomp = dumpInputStream;
}
decompressedInputStream = new CountingInputStream(
new BufferedInputStream(decomp, LOOKAHEAD));
}
private ExportSchemaVersion determineExportVersion() throws IOException
{
byte[] b = new byte[LOOKAHEAD];
decompressedInputStream.mark(LOOKAHEAD);
int read = decompressedInputStream.read(b, 0, LOOKAHEAD);
decompressedInputStream.reset();
String header = new String(b, 0, read);
if (header.contains("xmlns=\"http://www.mediawiki.org/xml/export-0.5/\""))
{
return ExportSchemaVersion.V0_5;
}
else if (header.contains("xmlns=\"http://www.mediawiki.org/xml/export-0.6/\""))
{
return ExportSchemaVersion.V0_6;
}
else if (header.contains("xmlns=\"http://www.mediawiki.org/xml/export-0.7/\""))
{
return ExportSchemaVersion.V0_7;
}
else if (header.contains("xmlns=\"http://www.mediawiki.org/xml/export-0.8/\""))
{
return ExportSchemaVersion.V0_8;
}
else if (header.contains("xmlns=\"http://www.mediawiki.org/xml/export-0.9/\""))
{
return ExportSchemaVersion.V0_9;
}
else if (header.contains("xmlns=\"http://www.mediawiki.org/xml/export-0.10/\""))
{
return ExportSchemaVersion.V0_10;
}
else
{
throw new IllegalArgumentException("Unknown xmlns");
}
}
/**
* The xerces UTF8Reader is broken. If the xerces XML parser is given an
* input stream, it will instantiate a reader for the encoding found in the
* XML file, a UTF8Reader in case of an UTF8 encoded XML file. Sadly, this
* UTF8Reader crashes for certain input (not sure why exactly).
*
* On the other hand, when given a reader, which is forced to work with a
* certain encoding, the xerces XML parser does not have this freedom and
* will apparently process Wikipedia dumps just fine.
*
* Therefore, in case you have trouble to parse a XML file, by specifying an
* encoding, you force the use of a Reader and can circumvent the crash.
*/
private XMLStreamReader getXmlStreamReader(Charset encoding) throws FactoryConfigurationError, XMLStreamException, UnsupportedEncodingException
{
XMLInputFactory xmlInputFactory = XMLInputFactory.newInstance();
if (encoding != null)
{
InputStreamReader isr = new InputStreamReader(decompressedInputStream, encoding);
return xmlInputFactory.createXMLStreamReader(isr);
}
else
{
return xmlInputFactory.createXMLStreamReader(decompressedInputStream);
}
}
private void setSchema(URL schemaUrl) throws SAXException, JAXBException
{
SchemaFactory sf = SchemaFactory.newInstance(
javax.xml.XMLConstants.W3C_XML_SCHEMA_NS_URI);
sf.setResourceResolver(new LSResourceResolverImplementation());
Schema schema = sf.newSchema(schemaUrl);
unmarshaller.setSchema(schema);
unmarshaller.setEventHandler(
new ValidationEventHandler()
{
public boolean handleEvent(ValidationEvent ve)
{
try
{
return DumpReader.this.handleEvent(ve, ve.getLocator());
}
catch (RuntimeException e)
{
throw e;
}
catch (Exception e)
{
throw new WrappedException(e);
}
}
});
}
private void installCallbacks()
{
final DumpReaderListener pageListener = new DumpReaderListener()
{
@Override
public void handlePage(Object mediaWiki, Object page)
{
try
{
DumpReader.this.handlePage(mediaWiki, page);
}
catch (RuntimeException e)
{
throw e;
}
catch (Exception e)
{
throw new WrappedException(e);
}
}
@Override
public boolean handleRevisionOrUploadOrLogitem(
Object page,
Object revision)
{
try
{
return DumpReader.this.handleRevision(page, revision);
}
catch (RuntimeException e)
{
throw e;
}
catch (Exception e)
{
throw new WrappedException(e);
}
}
};
unmarshaller.setListener(new Unmarshaller.Listener()
{
public void beforeUnmarshal(Object target, Object parent)
{
schemaVersion.setPageListener(target, pageListener);
}
public void afterUnmarshal(Object target, Object parent)
{
schemaVersion.setPageListener(target, pageListener);
}
});
}
private Unmarshaller createUnmarshaller(String contextPath) throws JAXBException
{
JAXBContext context = JAXBContext.newInstance(contextPath);
return context.createUnmarshaller();
}
}
DumpReaderListener.java 0000664 0000000 0000000 00000001735 13352705604 0046564 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble/wikitext/dumpreader /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.dumpreader;
public interface DumpReaderListener
{
void handlePage(Object mediaWiki, Object page);
/**
* @return Return true to add the item to the page's list or false if you
* handled it yourself.
*/
boolean handleRevisionOrUploadOrLogitem(Object page, Object revision);
}
DumpReaderWithHandler.java 0000664 0000000 0000000 00000003502 13352705604 0047202 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble/wikitext/dumpreader /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.dumpreader;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.Charset;
import javax.xml.bind.JAXBException;
import javax.xml.stream.FactoryConfigurationError;
import javax.xml.stream.XMLStreamException;
import org.slf4j.Logger;
import org.xml.sax.SAXException;
public class DumpReaderWithHandler
extends
DumpReader
{
private final DumpReaderListener listener;
// =========================================================================
public DumpReaderWithHandler(
InputStream is,
Charset encoding,
DumpReaderListener listener,
String url,
Logger logger,
boolean useSchema) throws JAXBException, FactoryConfigurationError, XMLStreamException, IOException, SAXException
{
super(is, encoding, url, logger, useSchema);
this.listener = listener;
}
// =========================================================================
@Override
protected void processPage(Object mediaWiki, Object page)
{
listener.handlePage(mediaWiki, page);
}
@Override
protected boolean processRevision(Object page, Object revision)
{
return listener.handleRevisionOrUploadOrLogitem(page, revision);
}
}
DumpUnmarshaller.java 0000664 0000000 0000000 00000026724 13352705604 0046316 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble/wikitext/dumpreader /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.dumpreader;
import java.io.File;
import java.io.InputStream;
import java.io.Reader;
import java.net.URL;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBElement;
import javax.xml.bind.JAXBException;
import javax.xml.bind.PropertyException;
import javax.xml.bind.Unmarshaller;
import javax.xml.bind.UnmarshallerHandler;
import javax.xml.bind.ValidationEventHandler;
import javax.xml.bind.annotation.adapters.XmlAdapter;
import javax.xml.bind.attachment.AttachmentUnmarshaller;
import javax.xml.stream.XMLEventReader;
import javax.xml.stream.XMLStreamReader;
import javax.xml.transform.Source;
import javax.xml.validation.Schema;
import javax.xml.validation.SchemaFactory;
import org.sweble.wikitext.dumpreader.model.DumpConverter;
import org.sweble.wikitext.dumpreader.model.Page;
import org.sweble.wikitext.dumpreader.model.Revision;
import org.w3c.dom.Node;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
public class DumpUnmarshaller
implements
Unmarshaller
{
public static final String CATALOG = "/catalog.xml";
private final Unmarshaller unmarshaller;
private final ExportSchemaVersion version;
private final JAXBContext context;
private DumpConverter converter;
// =========================================================================
public DumpUnmarshaller(ExportSchemaVersion version, boolean enableSchema) throws JAXBException, SAXException
{
this.version = version;
this.context = JAXBContext.newInstance(
version.getContextPath());
this.unmarshaller = context.createUnmarshaller();
if (enableSchema)
setSchema(getClass().getResource(version.getFragmentsSchema()));
}
// =========================================================================
public ExportSchemaVersion getVersion()
{
return version;
}
public JAXBContext getContext()
{
return context;
}
public Unmarshaller getUnmarshaller()
{
return unmarshaller;
}
public DumpConverter getConverter()
{
if (converter == null)
converter = new DumpConverter();
return converter;
}
public void setSchema(URL schemaUrl) throws SAXException
{
if (schemaUrl == null)
throw new IllegalArgumentException();
SchemaFactory sf = SchemaFactory
.newInstance(javax.xml.XMLConstants.W3C_XML_SCHEMA_NS_URI);
sf.setResourceResolver(new LSResourceResolverImplementation());
unmarshaller.setSchema(sf.newSchema(schemaUrl));
}
// =========================================================================
public T unmarshalToPageType(Node node, Class pageType) throws JAXBException
{
if (pageType != version.getPageType())
throw new IllegalArgumentException("Expected pageType == " + version.getPageType().getName());
@SuppressWarnings("unchecked")
T value = (T) unmarshaller.unmarshal(node, version.getPageType()).getValue();
return value;
}
public T unmarshalToPageType(Source source, Class pageType) throws JAXBException
{
if (pageType != version.getPageType())
throw new IllegalArgumentException("Expected pageType == " + version.getPageType().getName());
@SuppressWarnings("unchecked")
T value = (T) unmarshaller.unmarshal(source, version.getPageType()).getValue();
return value;
}
public T unmarshalToPageType(XMLStreamReader reader, Class pageType) throws JAXBException
{
if (pageType != version.getPageType())
throw new IllegalArgumentException("Expected pageType == " + version.getPageType().getName());
@SuppressWarnings("unchecked")
T value = (T) unmarshaller.unmarshal(reader, version.getPageType()).getValue();
return value;
}
public T unmarshalToPageType(XMLEventReader reader, Class pageType) throws JAXBException
{
if (pageType != version.getPageType())
throw new IllegalArgumentException("Expected pageType == " + version.getPageType().getName());
@SuppressWarnings("unchecked")
T value = (T) unmarshaller.unmarshal(reader, version.getPageType()).getValue();
return value;
}
// =========================================================================
public Page unmarshalToPage(Node node) throws JAXBException
{
return getConverter().convertPage(
unmarshaller.unmarshal(node, version.getPageType()).getValue());
}
public Page unmarshalToPage(Source source) throws JAXBException
{
return getConverter().convertPage(
unmarshaller.unmarshal(source, version.getPageType()).getValue());
}
public Page unmarshalToPage(XMLStreamReader reader) throws JAXBException
{
return getConverter().convertPage(
unmarshaller.unmarshal(reader, version.getPageType()).getValue());
}
public Page unmarshalToPage(XMLEventReader reader) throws JAXBException
{
return getConverter().convertPage(
unmarshaller.unmarshal(reader, version.getPageType()).getValue());
}
// =========================================================================
public T unmarshalToRevisionType(Node node, Class revisionType) throws JAXBException
{
if (revisionType != version.getRevisionType())
throw new IllegalArgumentException("Expected revisionType == " + version.getRevisionType().getName());
@SuppressWarnings("unchecked")
T value = (T) unmarshaller.unmarshal(node, version.getRevisionType()).getValue();
return value;
}
public T unmarshalToRevisionType(Source source, Class revisionType) throws JAXBException
{
if (revisionType != version.getRevisionType())
throw new IllegalArgumentException("Expected revisionType == " + version.getRevisionType().getName());
@SuppressWarnings("unchecked")
T value = (T) unmarshaller.unmarshal(source, version.getRevisionType()).getValue();
return value;
}
public T unmarshalToRevisionType(
XMLStreamReader reader,
Class revisionType) throws JAXBException
{
if (revisionType != version.getRevisionType())
throw new IllegalArgumentException("Expected revisionType == " + version.getRevisionType().getName());
@SuppressWarnings("unchecked")
T value = (T) unmarshaller.unmarshal(reader, version.getRevisionType()).getValue();
return value;
}
public T unmarshalToRevisionType(
XMLEventReader reader,
Class revisionType) throws JAXBException
{
if (revisionType != version.getRevisionType())
throw new IllegalArgumentException("Expected revisionType == " + version.getRevisionType().getName());
@SuppressWarnings("unchecked")
T value = (T) unmarshaller.unmarshal(reader, version.getRevisionType()).getValue();
return value;
}
// =========================================================================
public Revision unmarshalToRevision(Node node) throws JAXBException
{
return getConverter().convertRevision(
unmarshaller.unmarshal(node, version.getRevisionType()).getValue());
}
public Revision unmarshalToRevision(Source source) throws JAXBException
{
return getConverter().convertRevision(
unmarshaller.unmarshal(source, version.getRevisionType()).getValue());
}
public Revision unmarshalToRevision(XMLStreamReader reader) throws JAXBException
{
return getConverter().convertRevision(
unmarshaller.unmarshal(reader, version.getRevisionType()).getValue());
}
public Revision unmarshalToRevision(XMLEventReader reader) throws JAXBException
{
return getConverter().convertRevision(
unmarshaller.unmarshal(reader, version.getRevisionType()).getValue());
}
// =========================================================================
public Object unmarshal(File f) throws JAXBException
{
return unmarshaller.unmarshal(f);
}
public Object unmarshal(InputStream is) throws JAXBException
{
return unmarshaller.unmarshal(is);
}
public Object unmarshal(Reader reader) throws JAXBException
{
return unmarshaller.unmarshal(reader);
}
public Object unmarshal(URL url) throws JAXBException
{
return unmarshaller.unmarshal(url);
}
public Object unmarshal(InputSource source) throws JAXBException
{
return unmarshaller.unmarshal(source);
}
public Object unmarshal(Node node) throws JAXBException
{
return unmarshaller.unmarshal(node);
}
public JAXBElement unmarshal(Node node, Class declaredType) throws JAXBException
{
return unmarshaller.unmarshal(node, declaredType);
}
public Object unmarshal(Source source) throws JAXBException
{
return unmarshaller.unmarshal(source);
}
public JAXBElement unmarshal(Source source, Class declaredType) throws JAXBException
{
return unmarshaller.unmarshal(source, declaredType);
}
public Object unmarshal(XMLStreamReader reader) throws JAXBException
{
return unmarshaller.unmarshal(reader);
}
public JAXBElement unmarshal(
XMLStreamReader reader,
Class declaredType) throws JAXBException
{
return unmarshaller.unmarshal(reader, declaredType);
}
public Object unmarshal(XMLEventReader reader) throws JAXBException
{
return unmarshaller.unmarshal(reader);
}
public JAXBElement unmarshal(
XMLEventReader reader,
Class declaredType) throws JAXBException
{
return unmarshaller.unmarshal(reader, declaredType);
}
public UnmarshallerHandler getUnmarshallerHandler()
{
return unmarshaller.getUnmarshallerHandler();
}
/**
* @deprecated since JAXB2.0, please see
* {@link #setSchema(javax.xml.validation.Schema)}
*/
public void setValidating(boolean validating) throws JAXBException
{
throw new UnsupportedOperationException();
}
/**
* @deprecated since JAXB2.0, please see
* {@link #setSchema(javax.xml.validation.Schema)}
*/
public boolean isValidating() throws JAXBException
{
throw new UnsupportedOperationException();
}
public void setEventHandler(ValidationEventHandler handler) throws JAXBException
{
unmarshaller.setEventHandler(handler);
}
public ValidationEventHandler getEventHandler() throws JAXBException
{
return unmarshaller.getEventHandler();
}
public void setProperty(String name, Object value) throws PropertyException
{
unmarshaller.setProperty(name, value);
}
public Object getProperty(String name) throws PropertyException
{
return unmarshaller.getProperty(name);
}
public void setSchema(Schema schema)
{
unmarshaller.setSchema(schema);
}
public Schema getSchema()
{
return unmarshaller.getSchema();
}
public void setAdapter(@SuppressWarnings("rawtypes") XmlAdapter adapter)
{
unmarshaller.setAdapter(adapter);
}
@SuppressWarnings("rawtypes")
public void setAdapter(Class type, A adapter)
{
unmarshaller.setAdapter(type, adapter);
}
@SuppressWarnings("rawtypes")
public A getAdapter(Class type)
{
return unmarshaller.getAdapter(type);
}
public void setAttachmentUnmarshaller(AttachmentUnmarshaller au)
{
unmarshaller.setAttachmentUnmarshaller(au);
}
public AttachmentUnmarshaller getAttachmentUnmarshaller()
{
return unmarshaller.getAttachmentUnmarshaller();
}
public void setListener(Listener listener)
{
unmarshaller.setListener(listener);
}
public Listener getListener()
{
return unmarshaller.getListener();
}
}
ExportSchemaVersion.java 0000664 0000000 0000000 00000015530 13352705604 0046774 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble/wikitext/dumpreader /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.dumpreader;
import java.lang.reflect.Method;
import de.fau.cs.osr.utils.WrappedException;
public enum ExportSchemaVersion
{
V0_5
{
@Override
public String getSchema()
{
return "/export-0.5.xsd";
}
@Override
public String getFragmentsSchema()
{
return "/export-0.5-fragments.xsd";
}
@Override
public String getContextPath()
{
return "org.sweble.wikitext.dumpreader.export_0_5";
}
@Override
public String getMediaWikiNamespace()
{
return "http://www.mediawiki.org/xml/export-0.5/";
}
@Override
public Class getMediaWikiType()
{
return org.sweble.wikitext.dumpreader.export_0_5.MediaWikiType.class;
}
@Override
public Class getPageType()
{
return org.sweble.wikitext.dumpreader.export_0_5.PageType.class;
}
@Override
public Class getRevisionType()
{
return org.sweble.wikitext.dumpreader.export_0_5.RevisionType.class;
}
},
// =========================================================================
V0_6
{
@Override
public String getSchema()
{
return "/export-0.6-fixed.xsd";
}
@Override
public String getFragmentsSchema()
{
return "/export-0.6-fixed-fragments.xsd";
}
@Override
public String getContextPath()
{
return "org.sweble.wikitext.dumpreader.export_0_6";
}
@Override
public String getMediaWikiNamespace()
{
return "http://www.mediawiki.org/xml/export-0.6/";
}
@Override
public Class getMediaWikiType()
{
return org.sweble.wikitext.dumpreader.export_0_6.MediaWikiType.class;
}
@Override
public Class getPageType()
{
return org.sweble.wikitext.dumpreader.export_0_6.PageType.class;
}
@Override
public Class getRevisionType()
{
return org.sweble.wikitext.dumpreader.export_0_6.RevisionType.class;
}
},
// =========================================================================
V0_7
{
@Override
public String getSchema()
{
return "/export-0.7.xsd";
}
@Override
public String getFragmentsSchema()
{
return "/export-0.7-fragments.xsd";
}
@Override
public String getContextPath()
{
return "org.sweble.wikitext.dumpreader.export_0_7";
}
@Override
public String getMediaWikiNamespace()
{
return "http://www.mediawiki.org/xml/export-0.7/";
}
@Override
public Class getMediaWikiType()
{
return org.sweble.wikitext.dumpreader.export_0_7.MediaWikiType.class;
}
@Override
public Class getPageType()
{
return org.sweble.wikitext.dumpreader.export_0_7.PageType.class;
}
@Override
public Class getRevisionType()
{
return org.sweble.wikitext.dumpreader.export_0_7.RevisionType.class;
}
},
// =========================================================================
V0_8
{
@Override
public String getSchema()
{
return "/export-0.8.xsd";
}
@Override
public String getFragmentsSchema()
{
return "/export-0.8-fragments.xsd";
}
@Override
public String getContextPath()
{
return "org.sweble.wikitext.dumpreader.export_0_8";
}
@Override
public String getMediaWikiNamespace()
{
return "http://www.mediawiki.org/xml/export-0.8/";
}
@Override
public Class getMediaWikiType()
{
return org.sweble.wikitext.dumpreader.export_0_8.MediaWikiType.class;
}
@Override
public Class getPageType()
{
return org.sweble.wikitext.dumpreader.export_0_8.PageType.class;
}
@Override
public Class getRevisionType()
{
return org.sweble.wikitext.dumpreader.export_0_8.RevisionType.class;
}
},
// =========================================================================
V0_9
{
@Override
public String getSchema()
{
return "/export-0.9.xsd";
}
@Override
public String getFragmentsSchema()
{
return "/export-0.9-fragments.xsd";
}
@Override
public String getContextPath()
{
return "org.sweble.wikitext.dumpreader.export_0_9";
}
@Override
public String getMediaWikiNamespace()
{
return "http://www.mediawiki.org/xml/export-0.9/";
}
@Override
public Class getMediaWikiType()
{
return org.sweble.wikitext.dumpreader.export_0_9.MediaWikiType.class;
}
@Override
public Class getPageType()
{
return org.sweble.wikitext.dumpreader.export_0_9.PageType.class;
}
@Override
public Class getRevisionType()
{
return org.sweble.wikitext.dumpreader.export_0_9.RevisionType.class;
}
},
// =========================================================================
V0_10
{
@Override
public String getSchema()
{
return "/export-0.10.xsd";
}
@Override
public String getFragmentsSchema()
{
return "/export-0.10-fragments.xsd";
}
@Override
public String getContextPath()
{
return "org.sweble.wikitext.dumpreader.export_0_10";
}
@Override
public String getMediaWikiNamespace()
{
return "http://www.mediawiki.org/xml/export-0.10/";
}
@Override
public Class getMediaWikiType()
{
return org.sweble.wikitext.dumpreader.export_0_10.MediaWikiType.class;
}
@Override
public Class getPageType()
{
return org.sweble.wikitext.dumpreader.export_0_10.PageType.class;
}
@Override
public Class getRevisionType()
{
return org.sweble.wikitext.dumpreader.export_0_10.RevisionType.class;
}
};
// =========================================================================
public abstract String getSchema();
public abstract String getFragmentsSchema();
public abstract String getContextPath();
public abstract String getMediaWikiNamespace();
public abstract Class getMediaWikiType();
public abstract Class getPageType();
public abstract Class getRevisionType();
// =========================================================================
public void setPageListener(Object target, DumpReaderListener listener)
{
try
{
Method m = null;
if (getMediaWikiType().isInstance(target))
{
m = getMediaWikiType().getMethod(
"setPageListener",
DumpReaderListener.class);
}
else if (getPageType().isInstance(target))
{
m = getPageType().getMethod(
"setRevisionListener",
DumpReaderListener.class);
}
if (m != null)
m.invoke(target, listener);
}
catch (Exception e)
{
throw new WrappedException(e);
}
}
}
LSResourceResolverImplementation.java 0000664 0000000 0000000 00000010405 13352705604 0051476 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble/wikitext/dumpreader /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.dumpreader;
import java.io.InputStream;
import java.io.Reader;
import org.w3c.dom.ls.LSInput;
import org.w3c.dom.ls.LSResourceResolver;
final class LSResourceResolverImplementation
implements
LSResourceResolver
{
@Override
public LSInput resolveResource(
String type,
String namespaceURI,
String publicId,
final String systemId,
final String baseURI)
{
if ("http://www.w3.org/2001/XMLSchema".equals(type)
&& (publicId == null)
&& isExportXsd(getClass(), systemId))
{
return new LSInputImplementation(
getExportXsdFileNameIfExists(getClass(), systemId),
systemId,
baseURI);
}
else if ("http://www.w3.org/2001/XMLSchema".equals(type)
&& (publicId == null)
&& "http://www.w3.org/2001/xml.xsd".equals(systemId))
{
return new LSInputImplementation(
"/xml.xsd",
systemId,
baseURI);
}
else
{
System.err.println(String.format(
"Cannot resolve: type = '''%s''', namespaceURI = '''%s''', publicId = '''%s''', systemId = '''%s''', baseURI = '''%s'''",
type,
namespaceURI,
publicId,
systemId,
baseURI));
return null;
}
}
private static boolean isExportXsd(Class clazz, String systemId)
{
return (getExportXsdFileNameIfExists(clazz, systemId) != null);
}
private static String getExportXsdFileNameIfExists(
Class clazz,
String systemId)
{
if (systemId != null)
{
String string = "http://www.mediawiki.org/xml";
if (systemId.startsWith(string + "/export-")
&& systemId.endsWith(".xsd"))
{
String fileName = systemId.substring(string.length());
if (clazz.getResource(fileName) != null)
return fileName;
}
}
return null;
}
// =========================================================================
private final class LSInputImplementation
implements
LSInput
{
private final String xsdPath;
private final String systemId;
private final String baseURI;
private LSInputImplementation(
String xsdPath,
String systemId,
String baseURI)
{
this.xsdPath = xsdPath;
this.systemId = systemId;
this.baseURI = baseURI;
}
@Override
public void setSystemId(String systemId)
{
throw new UnsupportedOperationException();
}
@Override
public void setStringData(String stringData)
{
throw new UnsupportedOperationException();
}
@Override
public void setPublicId(String publicId)
{
throw new UnsupportedOperationException();
}
@Override
public void setEncoding(String encoding)
{
throw new UnsupportedOperationException();
}
@Override
public void setCharacterStream(Reader characterStream)
{
throw new UnsupportedOperationException();
}
@Override
public void setCertifiedText(boolean certifiedText)
{
throw new UnsupportedOperationException();
}
@Override
public void setByteStream(InputStream byteStream)
{
throw new UnsupportedOperationException();
}
@Override
public void setBaseURI(String baseURI)
{
throw new UnsupportedOperationException();
}
@Override
public String getSystemId()
{
return systemId;
}
@Override
public String getStringData()
{
return null;
}
@Override
public String getPublicId()
{
return null;
}
@Override
public String getEncoding()
{
return "UTF-8";
}
@Override
public Reader getCharacterStream()
{
return null;
}
@Override
public boolean getCertifiedText()
{
return false;
}
@Override
public InputStream getByteStream()
{
return getClass().getResourceAsStream(xsdPath);
}
@Override
public String getBaseURI()
{
return baseURI;
}
}
}
export_0_10/ 0000775 0000000 0000000 00000000000 13352705604 0044215 5 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble/wikitext/dumpreader MediaWikiTypeAspect.aj 0000664 0000000 0000000 00000002231 13352705604 0050374 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble/wikitext/dumpreader/export_0_10 /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.dumpreader.export_0_10;
import java.util.ArrayList;
import org.sweble.wikitext.dumpreader.DumpReaderListener;
public aspect MediaWikiTypeAspect
{
public void MediaWikiType.setPageListener(final DumpReaderListener listener)
{
page = (listener == null) ? null : new ArrayList()
{
private static final long serialVersionUID = 1L;
public boolean add(PageType page)
{
listener.handlePage(MediaWikiType.this, page);
return false;
}
};
}
}
PageTypeAspect.aj 0000664 0000000 0000000 00000003616 13352705604 0047415 0 ustar 00root root 0000000 0000000 sweble-wikitext-sweble-wikitext-3.1.9/sweble-wikitext-components-parent/swc-dumpreader/src/main/java/org/sweble/wikitext/dumpreader/export_0_10 /**
* Copyright 2011 The Open Source Research Group,
* University of Erlangen-Nürnberg
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.sweble.wikitext.dumpreader.export_0_10;
import java.util.ArrayList;
import org.sweble.wikitext.dumpreader.DumpReaderListener;
public aspect PageTypeAspect
{
public void PageType.setRevisionListener(final DumpReaderListener listener)
{
DumpReaderListener oldListener = null;
if ((this.revisionOrUpload != null) && (this.revisionOrUpload instanceof MyArrayList))
oldListener = ((MyArrayList) this.revisionOrUpload).getListener();
if (listener != null)
{
if (oldListener != listener)
{
this.revisionOrUpload = new MyArrayList(PageType.this, listener);
}
}
else
{
this.revisionOrUpload = null;
}
}
public static final class MyArrayList
extends
ArrayList